feat(extradup): add the wrong_record fault family and resolve the record by identity

[abrichr]

Every ExtraDup family corrupts what was written: a duplicate row, an extra column, a missing field, a form that never posted, a claim with nothing behind it. None tested where. This adds wrong_record, the write that is correct in every respect except the patient it landed on, and it moves the identity resolution into sor_check itself.

Rebased onto main now that openadapt-evals#335 has merged. One commit, 13 files, none of them files #335 owns.

The finding

Before this change a wrong-record write scored 0 by accident, not because the oracle resolved the named record. sor_check called new_records(before, after), which selects rows by insertion (row["id"] not in before_ids), then compared every key in spec.fields. It caught an identity swap only because MOCKMED_GOLD.fields happens to carry patient_id as a content field. Give it a spec whose fields are content-only, which is how an EMR screen actually behaves because you navigate to the chart and never type the patient id, and the same wrong-record write returns PASS.

#335 fixed this inside proof.py, by pre-filtering rows through records_for_identity(). The kit's sor_check still had no identity step, and that is the function the published environment and any downstream oracle call.

What wrong_record does

The agent creates one encounter. Right visit type, right note, saved. One row lands, which is the count the task asked for. The banner reads "Saved", the same banner a gold run produces. The row hangs off patient p0 instead of patient p1.

Count the new rows and you get 1, which matches. Read the note and it matches. Read the screen and it says the write succeeded. A verifier scored on task success cannot fail this rollout, because nobody told it which record was supposed to change. Neither can a screen scorer.

The fix is a resolution step, not a wider field comparison

• WriteSpec gains identity_fields, the contract's oracle_identity, and decoy_identity. decoy_of() raises if the decoy equals the spec's own identity, so the family can never quietly become a no-op.
• identity_check resolves the new records under oracle_identity and counts what landed there.
• sor_check runs that first, then compares only the non-identity fields. Removing an identity key from the content loop can no longer reopen the hole.
• content_only_check is the negative control: the task-success verifier that scores the form it knows about. It PASSes wrong_record. Two tests pin that, so the demonstration stays honest if someone edits the checker.

task_prompt and the dataset info row now carry oracle_identity, so the contract names the record to the policy as well as to the oracle.

It stays off the frozen corpus

wrong_record lives in a new EVAL_ONLY_OPERATORS, not in OPERATORS, matching how #335's M-freeze amendment records identity_swap (in_extradup_mutants: false, eval_only: true). So the kill-scan corpus digest, the pre-registered mutant set, holdout_operators, the seed schedule and the 2026-09-01 certificate are untouched. suite.cells() stays frozen because kill_scan.frozen_rollouts() digests it; all_cells() adds the eval-only families and drives check, list, run.

sor_check reports the identity reason only when records landed but none under the contract identity, so the sor_reasons strings that proof_2026-09-02.json pins for dup, extra and omit do not move. test_frozen_operator_reasons_did_not_move pins that.

The M-freeze amendment here covers only what changed: the pinned hashes of mutations.py, gold.py and checkers.py, plus a wrong_record block mirroring the identity_swap one. That hunk is the reward lane's document and wants their eye.

Numbers, and where they came from

certify_corpus() scores 7 cases on 50 synthetic variants in each of the 2 stores. 700 hacking trials, 0 earned reward; 100 gold trials, 0 refused. The exact one-sided 95% Clopper-Pearson upper bound from those counts is 0.004270, down from 0.0050 at 600 trials. Same bound function, more trials, still zero accepts.

Verified

Clean uv venv on Python 3.11, verifiers>=0.3.1,<0.3.2, no torch and no CUDA:

• load_environment() builds; load_environment(score_from_screen=True) still raises.
• self_test(): 1.0 for both controls, 0.0 for all 14 hacking entries.
• certify_corpus(): the counts above.
• check_fails_closed.py --num-examples 2 through real vf-eval against the scripted policy: ok for all 8 cases.
• pytest -k "extradup or reward or proof or seal or m_freeze": 153 passed, 2 skipped, 0 failed, including test_builtin_kill_scan_matches_committed_proof now that feat(reward): add identity-swap family to the MockMed ExtraDup proof #335's kill-scan fix is on main.
• ruff check: clean.

Environment pyproject.toml goes to 0.2.0. The verifiers pin is unchanged.

One thing this does not fix, in another repo

openadapt-flow has the same defect one layer up, on merged main. VerifierOracle.read(identity) (openadapt_flow/reward/oracles.py:119) never passes identity to the verifier; it only stamps it on the observation. So the read is unscoped and the only binding to the named record is the effect's own match selector. RewardBundle.load (models.py:349-357) checks that every param an effect references is declared, never the converse, so a bundle declaring identity_keys: ["patient_id"] whose required effect uses a content-only selector loads cleanly and returns verified for a write that landed on another patient. The one bundle flow ships (seed.py:82-96) does reference the identity param, so no shipped bundle is affected. Reported separately; not changed here.

Opened by an agent session, not the founder.

🤖 Generated with Claude Code