feat(str): corpus-wide Y-STR marker report + mutation-rate provenance

[JamesKane]

What

Adds a corpus-wide Y-STR marker report — GET /str-markers (HTML) and GET /api/v1/reports/str-markers (JSON): the observed repeat range (min / modal / max) of every Y-STR marker across all profiles we hold, joined to its repeat motif and mutation rate.

Until now the STR data was only ever read vertically — du_db::ystr reads it per-haplogroup for modal signatures, ASR and branch ages — so there was no way to ask what the corpus says about a given marker. This is the horizontal view, the shape a vendor project page publishes as its MIN/MAX/MODE row, over a larger cohort.

Corpus-wide only; the clade filter can be added later without a schema change.

Migrations

0069_str_marker.sql — genomics.str_marker, the first STR marker reference in the schema. Marker identity was previously a bare marker_name TEXT in every table; nothing said what DYS390 is. Seeded to 142 rows (125 with a motif) by parsing the free-text str_mutation_rate.source, the only motif information this database holds. coordinates mirrors the core.genome_region shape so a locus source can be loaded in place later — it is empty today, as no table has ever held an STR locus position.

0070_str_rate_provenance.sql — re-keys str_mutation_rate from marker_name UNIQUE to (marker_name, method), plus a partial unique index on the single active rate per marker, and records per-branch which rates produced an age estimate.

Three findings that shaped the design

1. Marker-name aliasing is load-bearing, not defensive. Profiles say DYS19 and Y-GATA-H4; the rate table says DYS394 and YGATAH4 for the same loci. A join on name alone silently drops the two most recognizable markers in the panel. DYS19/DYS394 are also duplicate rate rows for one locus (identical rate and motif) which collide on insert, so the seed dedupes them. Every lookup folds through str_marker.aliases.

2. ~86% of the STR age model runs on a placeholder rate. Only 114 of 829 observed markers have a published rate; the rest fall back to DEFAULT_STR_RATE (0.0025) at ystr.rs:397/:627. Measured across all 1,963 STR_VARIANCE branches:

	marker-scorings
measured rate	221,819 (13.7%)
default rate	1,393,730 (86.3%)

This was previously invisible. It is now recorded per branch (rate_method, measured_rate_markers, default_rate_markers) and shown per marker.

3. Null alleles exist in multi-copy form too. A reported 0 is FTDNA's deleted-locus convention, not a zero repeat count — including it makes DYS448 read min 0 instead of 17, and CDY read min 0-0. Any zero copy in a vector counts as a null allele and is excluded from min/modal/max. Only fully-null 0-0 vectors actually occur today.

Forward capacity (designed, NOT built here)

load_marker_models now reads WHERE is_active, so promoting a derived rate is a data operation rather than a code change. The ingredients for deriving our own rates already exist: tree.haplogroup_str_asr holds 9.4M reconstructed states across 823 markers, and 10,242 haplogroups carry both ASR and a SNP_POISSON age — ~7× the marker coverage of our published rates.

Important

When that job is written, branch lengths must come from method='SNP_POISSON' ages only — never COMBINED or STR_VARIANCE, which would feed STR-derived ages back into STR rate estimation and let the model converge on itself. This is the one part of the design that fails silently if got wrong, so the job should assert the filter rather than parameterize it.

Verification

• 829 markers, 514,327 observations; 107 with a motif, 114 with a measured rate.
• DYS448 min 17, not 0, with 1 null allele · DYS390 21/24/26 · DYS385 modal combination 11-15 · DYS19 and Y-GATA-H4 resolve motif/rate only via alias folding · DYS447 reads DEFAULT_RATE.
• measured_rate_markers + default_rate_markers = marker_count on every row (0 mismatches).
• run-once branch-age re-run produced str_ages=1963 snp=10242 combined=10257 — identical to the pre-change baseline, so the is_active filter causes no regression.
• 38 du-db + 65 du-web tests pass (--test-threads=1), including 2 new integration tests and 4 new unit tests.

🤖 Generated with Claude Code