feat: Health Score v2 (Akrites) scoring pipeline, lifecycle, and impact score (CM-IN-1196)

[gaspergrom]

Summary

Adds the Health Score v2 (Akrites methodology) scoring pipeline: per-repo maintainer health, security/supply-chain, and development-activity category scores, plus a lifecycle classifier and an impact score, rolled up to project level. Also fixes several correctness bugs found during data verification against production before rollout.

• Split Health Score v2 computation into 5 independently-materialized category pipes (maintainer/security/development/lifecycle/impact) to keep copy-job runtime down — a single combined pipe was re-scanning base tables per category and had grown from ~13min to 20-28+ min per run.
• healthScoreV2 applies spec-defined graceful degradation: if 1-2 of 3 categories are unavailable, the score rescales across only the available categories rather than defaulting to a fabricated 0; NULL only when all three are unavailable.
• Bus factor fix: the maintainer-health bus-factor score counted maintainer role records purely by an unexpired endDate (a manually-curated field that stays at a "still active" sentinel indefinitely unless someone explicitly closes the role). This let archived/dead repos keep a maxed-out bus-factor score from maintainers with no real activity in years, which combined with the graceful-degradation rescale to produce healthScoreV2 = 100 on dead repos. Bus factor now also requires the maintainer to have actual activityRelations activity on that repo within the same trailing 12-month window. Verified against production: Xamarin.Forms and SpaceVim (archived) drop from a maxed bus-factor score to 0; kubernetes/kubernetes stays healthy and unaffected.
• Lifecycle fallback fix: the lifecycle decision tree's final ELSE branch always resolved to 'active' whenever a repo failed every other branch, including repos with zero commits, zero issues, and zero PRs in every window checked (confirmed live on a fully dormant, non-archived repo with no commit-timeline data and no signal at all). The chain now returns NULL for that case instead of a fabricated 'active', propagated correctly through:
  • health_score_v2_lifecycle_ds — widened lifecycleLabelV2 from String to Nullable(String) (was going to hard-fail the copy job on any NULL insert).
  • health_score_v2.pipe — removed a coalesce(lifecycleLabelV2, 'active') that would have silently erased the NULL back into 'active' one hop downstream.
  • project_insights_copy.pipe — the project-level "best-state-wins" rollup already drops NULL entries for free via groupArray, but an all-NULL or fully-excluded project produces an empty array, and arrayElement([], 1) on an empty array silently returns '' rather than NULL. Added an explicit guard. While tracing this, found the actual root cause of 5 pre-existing blank-lifecycleLabel projects (each project's sole repo has repositories.excluded = true) — a separate, pre-existing data situation, not fixed here, but no longer surfaces as a silent empty string.

All fixes verified against live production Tinybird data before merge, including full-chain propagation checks and regression checks against known-healthy projects (kubernetes, projects with real but thin activity signal).

Test plan

• tb check passes on all changed pipes/datasources
• Verified bus-factor fix against production data: archived repos (Xamarin.Forms, SpaceVim, HPCToolkit, Ursa) drop from healthScoreV2=100 to realistic scores (22-62); active repos (kubernetes, GravityView, Hitomi-Downloader, binutils-gdb) unaffected
• Verified lifecycle NULL fix against production data: a genuinely zero-signal repo (blocknetdx/blocknet) resolves to NULL at both repo and project level; archived repos still correctly resolve to 'archived' even with zero activity; repos with any real signal still resolve to a real state, not NULL
• Verified datasource schema widening (String → Nullable(String)) is a non-breaking, metadata-only ClickHouse ALTER
• Deploy to staging, confirm pipes deploy cleanly (staging data is known to be sparse, so this validates deployability not data correctness)
• Deploy to production, re-validate full chain against live data post-deploy

Deploy notes

Deploy order matters — datasources before pipes, then pipes in dependency order:

1. health_score_v2_lifecycle_ds.datasource, health_score_v2_repo_copy_ds.datasource (schema changes)
2. health_score_v2_lifecycle.pipe, health_score_v2_maintainer.pipe (independent, both feed health_score_v2)
3. health_score_v2.pipe (reads both category datasources above)
4. project_insights_copy.pipe (reads health_score_v2_repo_copy_ds)

After pushing pipes, the copy jobs need a forced run (tbc <pipe> --yes) rather than waiting for the nightly cron, in the same order, so the NULL/bus-factor fixes propagate through the full chain before this is user-visible.

[cursor]

PR Summary

High Risk
Large new analytics pipeline and user-visible score/label fields; scoring logic changes can mis-rank projects until copy jobs run in deploy order.

Overview
Introduces the Akrites Health Score v2 Tinybird stack: five scheduled copy pipes (maintainer, security, development, lifecycle, impact) materialize per-repo category scores and labels, and health_score_v2.pipe joins them into health_score_v2_repo_copy_ds with Layer 2 rescaling when some categories are NULL instead of fabricating zeros.

Project insights gains healthScoreV2, healthLabel, lifecycleLabel, impactScore, and impactLabel on project_insights_copy_ds, rolled up in project_insights_copy.pipe (repo mean for health, best-state-wins lifecycle, max impact) and exposed from project_insights / project_repo_insights APIs.

Correctness fixes: bus factor now requires 12-month activityRelations activity, not only an open maintainer role (stops dead repos rescaling to healthScoreV2 = 100); lifecycle returns NULL when all activity signals are zero (not default 'active'); health_score_v2.pipe drops coalesce(..., 'active') on lifecycle; project lifecycle rollup guards empty groupArray so it returns NULL instead of ''.

^{Reviewed by Cursor Bugbot for commit a20d1bc. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

PR titles must follow Conventional Commits. Love from, Your reviewers ❤️.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit a20d1bc. Configure here.}

[cursor]

Bus factor counts non-contribution activity

Medium Severity

The bus-factor fix requires recent activityRelations rows on the repo, but the subquery does not restrict activity type. Stars, forks, and other non-contribution events can satisfy the join, so maintainers with no real maintenance work in the window may still inflate bus factor—similar to the stale role-record issue this change was meant to fix.

 

^{Triggered by learned rule: Tinybird pipes counting contributors must filter by activityTypes}

^{Reviewed by Cursor Bugbot for commit a20d1bc. Configure here.}

[cursor]

NULL health score becomes zero

High Severity

Project and repo rows set healthScoreV2 with toUInt8(round(hv2.healthScoreV2)) without a NULL guard, unlike upstream pipes that use toNullable. Missing or unavailable scores can become 0, which is a valid “critical” score and contradicts healthLabel when that field is NULL. When every included repo has NULL healthScoreV2, avg can yield NaN; NaN is not treated as NULL in the label multiIf, so the project can show healthLabel critical while the score is wrong.

Additional Locations (1)

• services/libs/tinybird/pipes/project_insights_copy.pipe#L288-L289

 

^{Reviewed by Cursor Bugbot for commit a20d1bc. Configure here.}

[Copilot]

Pull request overview

Adds the Akrites Health Score v2 pipeline and exposes repository/project health, lifecycle, and impact metrics.

Changes:

• Materializes five independent scoring categories and combines available health signals.
• Adds lifecycle and impact scoring with project-level rollups.
• Exposes the new metrics through insights endpoints and datasources.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
pipes/project_repo_insights.pipe	Exposes Akrites fields for projects and repositories.
pipes/project_insights.pipe	Exposes Akrites project fields.
pipes/project_insights_copy.pipe	Rolls up and labels repository scores.
pipes/health_score_v2.pipe	Combines category scores with graceful degradation.
pipes/health_score_v2_security.pipe	Computes security and supply-chain health.
pipes/health_score_v2_maintainer.pipe	Computes maintainer health.
pipes/health_score_v2_lifecycle.pipe	Classifies repository lifecycle.
pipes/health_score_v2_impact.pipe	Computes package-mediated impact.
pipes/health_score_v2_development.pipe	Computes development activity health.
datasources/project_insights_copy_ds.datasource	Adds Akrites insight columns.
datasources/health_score_v2_security_ds.datasource	Stores security scores.
datasources/health_score_v2_repo_copy_ds.datasource	Stores combined repository results.
datasources/health_score_v2_maintainer_ds.datasource	Stores maintainer scores.
datasources/health_score_v2_lifecycle_ds.datasource	Stores lifecycle classifications.
datasources/health_score_v2_impact_ds.datasource	Stores raw impact scores.
datasources/health_score_v2_development_ds.datasource	Stores development scores.

Comments suppressed due to low confidence (3)

services/libs/tinybird/pipes/health_score_v2_lifecycle.pipe:85

• All three values only use issues opened in the last 18 months, but the aggregation currently reads every partition in issues_analyzed. Add the 18-month predicate to WHERE so ClickHouse can prune old yearly partitions.

        FROM issues_analyzed
        GROUP BY channel

services/libs/tinybird/pipes/health_score_v2_lifecycle.pipe:90

• The metric only counts PRs from the last 18 months, yet this scans all yearly partitions in pull_requests_analyzed. Push the same time condition into WHERE to prune historical partitions.

        FROM pull_requests_analyzed
        GROUP BY channel

services/libs/tinybird/datasources/health_score_v2_repo_copy_ds.datasource:24

• This statement contradicts the implemented Layer 2 graceful degradation in health_score_v2.pipe: unavailable categories are excluded and remaining category scores are rescaled, while the result is NULL only when all categories are unavailable. The earlier “sum” description is also incomplete for that case.

    - No graceful-degradation/signal-coverage redistribution yet — a repo with zero signal in a category
      scores 0 there rather than having points redistributed from available categories.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.