perf(str): precompute the marker report instead of aggregating per request

[JamesKane]

Follow-up to #22. The /str-markers page is slow on production.

The measurement

It aggregated every stored profile on every request — ~1.9s over the 876-profile dev corpus, growing linearly with the corpus, so production is worse.

Profiling put the cost somewhere indexes cannot reach:

Stage	Time
JSONB expansion (seq scan + jsonb_array_elements, 514k observations)	36ms
Aggregate machinery (mode() WITHIN GROUP, 3× ordered array_agg, 2× DISTINCT)	~1,840ms
The two reference joins — the only index-servable part	13ms

marker_name is already a PK and the active-rate partial index already applies, so there was nothing to add. I tried the obvious rewrites before reaching for precomputation:

• Raise work_mem to 256MB (the plan spills 52MB to disk): 1,888ms — no change
• Drop count(DISTINCT sid): 1,556ms (−17%)
• Drop both DISTINCT aggregates: 1,385ms (−26%)
• Pre-aggregate by distinct value before the ordered-set aggregates (the textbook fix): 1,449ms (−23%)

None of that is enough, and none of it stops the growth.

The change

Precompute the rollup into genomics.str_marker_stat, refreshed by du-jobs run-once str-marker-stats — the same pattern as coverage-norms, discovery-consensus and tree-samples-recompute. A plain table rather than a MATERIALIZED VIEW: this schema uses none, and the refresh belongs inside the job's transaction.

	before	after
DB query	1,875ms	0.94ms
Page response	~2s	13ms

And it no longer scales with the corpus.

Design notes

• Full replace in one transaction. There is no incremental form — a single new profile can move a marker's min, max and mode at once — and readers see one whole snapshot or the other.
• Auto-refresh after an applied ftdna-str import, since a bulk load invalidates every row.
• Live fallback while the table is empty, covering the window between deploying migration 0071 and the first refresh. The report stays correct rather than going blank, and logs a warning naming the job instead of failing quietly.
• The page now shows when it was last recomputed, because the figures are no longer live.
• The aggregation SQL has one definition, shared by the refresh and the fallback. The integration test asserts both paths return identical rows, so they cannot drift.

Deploy

Run du-jobs run-once str-marker-stats after migrating. Forgetting it is not fatal — the page works at the old speed and says so in the log. Worth adding to the timer set for the federated path: fed.str_profile arrives continuously via Jetstream and nothing else triggers a refresh for it.

Verification

• cargo clippy --workspace --locked -- -D warnings clean
• du-db 114/114 against a fresh, empty database in parallel — the configuration that exercises migration 0071 from scratch
• du-web 65/65
• Served JSON byte-identical to pre-change on the spot-check markers (DYS19, DYS385, DYS390, DYS447, DYS448, CDY)

🤖 Generated with Claude Code