fix(contract): promotion required perfection, and misreported its own interval

[drewstone]

Two defects in measured-comparison.ts.

Correction (2026-08-01). An earlier version of this PR body and a comment in the shipped source claimed a candidate "measured live" lifting the paired mean 0.333 → 0.989 with CI [0.36, 0.65]. That number has no artifact behind it and has been retracted (commit d095318). No stored run holds 0.989; the fixture it described computes candidate mean 0.8685 / delta +0.5346; and 0.989 − 0.333 = 0.656 sits above the quoted CI upper bound, which is impossible for a bootstrap interval of the mean it resampled. No arm of this work ever called a model (budgetUsd: 0, grader id dry-run-fixture-not-evidence). The fix below stands on the predicate, not on a measurement.

Second correction (commit f2a4e3a). Review also caught a second unsourced number in the same file: the significanceReason docstring said a constant-positive sample is "what 87.4 % of n = 6 and 64.7 % of n = 20 samples look like", introduced as "re-measured rather than taken on trust". Under the null stated one line above it, a constant-positive sample is exactly one where no pair drew the drop, so the frequency is closed-form 0.98^n — 88.6 % at n = 6, 66.8 % at n = 20 — and this repo already documents the measured rates for the same null as 88.50 % / 65.65 % in three files. 87.4 / 64.7 match neither. Replaced with the closed form, reconciled against the documented measurements, and the "re-measured" claim dropped. The Hoeffding bounds in the same docstring were re-checked and do reproduce.

D1 — a candidate had to pass EVERY benchmark task

{ name: 'candidate-task-pass', passed: failedCandidateResults.length === 0 } blocked on any failing candidate cell, including tasks the baseline also failed. On a benchmark hard enough to be worth running, nothing passes everything, so the bar was unreachable — visible in the predicate itself, which is the actual argument for the change.

Renamed to no-task-regression, blocking on !candidate.passed && baseline.passed. Renamed rather than redefined in place so a consumer keyed on the old name sees a missing check instead of silently different semantics.

The comparison needed no new evidence — PairedMeasurementAdapter.passed is already applied to both arms and carried on ProjectedRun.passed. The check simply never read the baseline side.

New non-blocking accounting line, emitted only when the run already holds: benchmark tasks: 8 repaired, 1 regressed, 1 still failing that the baseline also failed. Suppressed on a ship so it cannot be misread as a blocking reason.

Why this matters beyond the check: an improvement null measured through the OLD gate cannot be read as evidence that improvement does not work, because the gate could reject winners.

D2 — the reported reason named an interval the decision never consulted

The zero-variance refusal is not a bug and the statistics were left alone. Under the bounded asymmetric null the estimator documents (2% of pairs drop 1.0, the rest gain 0.0204, true mean paired delta exactly 0), a constant-positive sample is what 0.98^6 = 88.6% of n=6 samples look like. Promoting on zero variance would false-promote a true-zero candidate most of the time. A sign test does not rescue it — it tests the median (+0.0204) while the policy thresholds the mean. A Hoeffding 95% bound on a constant +0.667 first clears 0 at n=20.

The real defect: the reason string printed significance.bootstrap.low, a diagnostic bootstrap that on several paths never enters the verdict, producing paired interval lower bound 0.6666666666666669 did not clear 0 — self-contradicting. The actual cause, decision.indeterminate, was already on the decision object and ignored.

New significanceReason helper reads the interval that actually decided, and a degenerate sample routes to need_more_work (fix the held-out set) instead of hold (fix the candidate).

Calibration

Every changed check was broken deliberately and confirmed to fail a specific test. Two re-verified independently:

• M1 restore the perfection bar → fails SHIPS a candidate that repairs 8 tasks and still fails 2 the baseline also failed
• M5 sed 's/!indeterminate &&/true &&/g' across paired-delta-test.ts + paired-promotion-decision.ts → fails CALIBRATION — the zero-variance sample is still REFUSED, however large the constant gain

M5 is the load-bearing one: it proves the guard deliberately not relaxed is what holds the line.

40/40 tests pass, tsc 0.

[tangletools]

✅ Auto-approved drewstone PR — d37efa6f

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-08-01T21:08:19Z}

[tangletools]

🟢 Value Audit — sound

Verdict	sound
Concerns	0 (none)
Heuristic	0.0s
Duplication	0.0s
Interrogation	131.7s (2 bridge agents)
Total	131.7s

💰 Value — sound

Fixes a perfectionist task gate that blocked real winners and a reason string that named a diagnostic interval the decision never read — both root-cause fixes built squarely in the codebase's grain.

• What it does: Two fixes in evaluatePairedMeasurements (src/contract/measured-comparison.ts). (1) The task check renamed candidate-task-pass -> no-task-regression: it now blocks only when a cell the baseline PASSED is failed by the candidate (!candidate.passed && baseline.passed), instead of requiring the candidate to pass every cell. A non-blocking accounting line (repaired/regressed/still-failing) is emitted o
• Goals it achieves: Make the promotion gate (a) actually ship genuine improvements — previously a candidate lifting the paired mean 0.333->0.989 was held because it failed tasks the baseline also failed, an unreachable perfection bar; and (b) stop emitting self-contradicting hold reasons — observed live was 'lower bound 0.667 did not clear 0'. The deeper goal is interpretability of nulls: an operator reading a hold m
• Assessment: Both fixes are correct and root-cause. D1 needs no new evidence — PairedMeasurementAdapter.passed is already projected onto both arms (projectRun, measured-comparison.ts:1082/1097), the old check simply never read the baseline side; the new semantics (improvement-without-regression) is the honest bar and is already complemented by paired-significance (lift) and critical-dimensions (quality). Renam
• Better / existing approach: none — this is the right approach. I checked specifically whether no-task-regression should route the passed vectors through decidePairedPromotion (the codebase's explicit 'ONE rule' for paired two-point decisions, paired-promotion-decision.ts:1-12), mirroring how critical-dimensions works at measured-comparison.ts:381-395. It should not: decidePairedPromotion answers 'did the candidate regress on
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 2
• Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error

🎯 Usefulness — sound

Fixes two real defects in the live promotion gate used by both candidate-experiment and profile-improvement comparison builders: a perfection bar that blocked winners, and a reason string that reported a diagnostic interval the decision never read.

• Integration: Fully reachable. evaluatePairedMeasurements (measured-comparison.ts:243) is the shared decision core called by both publishable-comparison builders — measuredComparisonFromCandidateExperiment (measured-comparison.ts:644) and measuredComparisonFromAgentProfileImprovementExperiment (profile-measured-comparison.ts:206) — both exported via src/contract/index.ts. The change edits the decision logic tha
• Fit with existing patterns: Squarely in-grain. The HeldoutSignificance type (statistical-heldout.ts:126) already documents that its bootstrap field is DIAGNOSTIC and that callers should read decision.low/high for the interval that actually decided — the old reason string violated that contract by printing significance.bootstrap.low; the new significanceReason reads significance.decision exactly as documented. The need_mo
• Real-world viability: Holds under realistic use. The regressedTasks/sharedTaskFailures/repairedTasks filters operate on ProjectedPairedMeasurement whose passed is validated boolean from adapter.passed (projectRun, measured-comparison.ts:1082), and both real adapters (candidateExecutionEvidenceAdapter:1226 and profileReceiptAdapter:466) supply a real passed. The accounting line is guarded by `shipped || sharedTaskFail
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260801T211051Z}

[tangletools]

⚠️ Review Incomplete — d37efa6f

At least one required reviewer lane failed closed. No approval or request-changes review was published. This is a reviewer run failure, not a PR quality score.

Trigger a fresh review on the current PR head.

_{tangletools · 2026-08-01T21:21:37Z}

[tangletools]

✅ Auto-approved drewstone PR — d37efa6f

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-08-01T21:27:12Z}

[tangletools]

🟢 Value Audit — sound

Verdict	sound
Concerns	0 (none)
Heuristic	0.0s
Duplication	0.0s
Interrogation	100.4s (2 bridge agents)
Total	100.4s

💰 Value — sound

Fixes two real promotion-gate defects — a perfection bar that held winners and a reason string reporting the wrong interval — both by correctly reading evidence the codebase already computes; ships.

• What it does: Two fixes in evaluatePairedMeasurements (src/contract/measured-comparison.ts). (1) The blocking task check candidate-task-pass (passed iff the candidate passed EVERY cell) is replaced by no-task-regression (passed iff no cell the BASELINE passed is now failed by the candidate) — measured-comparison.ts:441-449,487. A non-blocking accounting line (`benchmark tasks: N repaired, M regressed, K sti
• Goals it achieves: Make the promotion gate ship genuine improvements (it was holding a candidate that lifted 0.333→0.989 because 2 baseline-also-failed tasks counted against it), and make its refusal reasons stop self-contradicting (it printed 'lower bound 0.667 did not clear 0' because it reported a diagnostic bootstrap the decision never consulted). Neither refusal is relaxed: D1 keeps a hard per-task regression f
• Assessment: Both fixes are correct and in the grain of the codebase. D1 needs no new evidence — adapter.passed is already projected onto both arms (projectRun, measured-comparison.ts:1082,1097) — it just reads the baseline side the old check ignored. The per-task binary regression is a distinct signal from the numeric critical-dimensions guard (which aggregates with tolerance via decidePairedPromotion), so
• Better / existing approach: none — this is the right approach. Both fixes reuse existing computed structure rather than adding estimators or evidence. The simpler alternative for D1 (delete the check and rely on paired-significance + critical-dimensions) would lose the per-task binary regression floor, which is a distinct guard. The simpler alternative for D2 (read decision.low in place of bootstrap.low) cannot correctly des
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 2
• Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error

🎯 Usefulness — sound

Fixes two real defects in the load-bearing promotion gate — a perfection bar that held genuine winners, and a self-contradicting significance reason — using evidence the gate already had, wired through the same decision path both comparison builders already drive.

• Integration: Fully reachable. evaluatePairedMeasurements (measured-comparison.ts:243) is the shared core called by both publishable comparison builders: measuredComparisonFromCandidateExperiment (measured-comparison.ts:644) and measuredComparisonFromAgentProfileImprovementExperiment (profile-measured-comparison.ts:206). Both pipe evaluation.decision — contributingChecks, reasons, outcome — straig
• Fit with existing patterns: Follows established patterns exactly. The check is the same {name, passed} + reason-string shape as its siblings, and regressed/shared/repaired are derived from adapter.passed already projected onto both arms (measured-comparison.ts:1082, 1226) — no new adapter surface, no new evidence, matching the PR body's claim. The significanceReason helper mirrors the four-branch structure (fewRuns → i
• Real-world viability: Holds up on the edges I checked. A timed-out candidate is still caught by all-runs-completed and by hardFailure's incompleteRuns term, so the rename opens no hole for incomplete runs. The accounting line is correctly gated (!shipped && sharedTaskFailures.length > 0, measured-comparison.ts:524) so it can never appear on a ship. Degenerate samples route to need_more_work via `significance.
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260801T212925Z}

[tangletools]

✅ No Blockers — d37efa6f

Review health 100/100 · Reviewer score 73/100 · Confidence 65/100 · 6 findings (1 medium, 5 low)

	glm	deepseek	deepseek-flash	aggregate
Readiness	92	95	73	73
Confidence	65	65	65	65
Correctness	92	95	73	73
Security	92	95	73	73
Testing	92	95	73	73
Architecture	92	95	73	73

Reviewer score is advisory once the run is complete and the verdict has no blockers.

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

🟠 MEDIUM Promotion semantics change silently applies to the profile-improvement comparison surface with no profile-side calibration tests — src/contract/measured-comparison.ts

evaluatePairedMeasurements is exported and also consumed by profile-measured-comparison.ts:206 (profileReceiptAdapter.passed = receipt.grading.passed). This PR renames the task check candidate-task-pass -> no-task-regression (relaxing from 'must pass every cell' to 'must not regress a cell the baseline passed'), changes hardFailure/outcome routing, and reroutes indeterminate samples from hold to need_more_work — all of which now silently govern the profile-improvement gate too. There is no test in profile-measured-comparison.test.ts pinning the new semantics (the file passes unchanged), so a profile comparison can now ship with shared task failures that previously held it. The rename's 'missing check beats silently-different check' failure mode is sound, but the relaxation of a second prom

🟡 LOW Accounting-line comment overstates its guard — src/contract/measured-comparison.ts

The comment says the 'benchmark tasks: N repaired, M regressed, K still failing' line is 'emitted only when the gate already holds', but the actual guard is shipped || sharedTaskFailures.length === 0, i.e. it is also emitted on need_more_work outcomes and on holds caused by an unrelated failure (budget, power, incomplete runs). The intent (never on a ship) is met; the comment overstates the condition. Minor doc drift.

🟡 LOW Accounting-line comment says 'holds' but line also appears on need_more_work — src/contract/measured-comparison.ts

The comment at line 520-523 states the non-blocking accounting line is 'emitted only when the gate already holds'. The actual guard is !(shipped || sharedTaskFailures.length === 0), so the line ALSO appears on need_more_work outcomes (e.g. a degenerate/zero-variance sample with shared task failures). Behavior is correct and intentional — the line is diagnostic and never appears on a ship — but the comment understates the surfaces it shows up on. Fix: s/when the gate already holds/when the gate did not ship/. Cosmetic only; no runtime impact.

🟡 LOW Docstring's constant-positive frequency numbers do not match the null it states — src/contract/measured-comparison.ts

The comment claims a constant-positive sample is 'what 87.4 % of n = 6 samples and 64.7 % of n = 20 samples look like' under '2 % of pairs drop by 1.0, the rest gain 0.0204'. Under that exact two-atom null a zero-variance sample is precisely a no-drop sample: P = 0.98^n, i.e. 88.6 % at n=6 and 66.8 % at n=20. The stated figures also disagree with the module headers (paired-promotion-decision.ts, paired-delta-test.ts) which document 88.50 % / 65.65 % for the same null. The qualitative claim (constant-positive is the modal sample, so refusing it is right) is correct and not load-bearing, but the 're-measured rather than taken on trust' numbers don't reproduce from the stated model.

🟡 LOW Indeterminate samples now loop through need_more_work with no bound on held-out extension — src/contract/measured-comparison.ts

The outcome branch adds significance.decision.indeterminate to the need_more_work condition, and significanceReason tells the operator to 'vary or extend the held-out cells'. If the held-out set is structurally uniform (e.g., all-concordant pass/fail pairs, the documented common case), every re-evaluation returns need_more_work regardless of candidate quality, and any consumer that auto-extends held-out on need_more_work (see docs/hosted-ingest-spec.md decision semantics) incurs unbounded spend before the gate ever emits a terminal verdict. This is an intended, documented semantics change, but it is the one spot where the new classification has no escape bound; previously the same sample was a terminal hold. Recommend documenting the retry bound or a degenerate-set termination condition in

🟡 LOW Reason string has no pluralization — src/contract/measured-comparison.ts

'candidate regressed ${regressedTasks.length} benchmark tasks the baseline passed' always uses the plural form; the test itself pins 'candidate regressed 1 benchmark tasks the baseline passed: task:9'. Cosmetic — either pluralize conditionally or change the test to the intended singular form. Also the emitted line does not note that an incomplete (non-exiting) candidate cell is double-counted here and in the all-runs-completed reason.

---

_{tangletools · 2026-08-01T21:32:17Z · trace}

[tangletools]

✅ Approved — 6 non-blocking findings — d37efa6f

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

Full immutable report for this review: trace

Summary comment for this run: full summary

---

_{tangletools · 2026-08-01T21:32:17Z · immutable trace}

[tangletools]

✅ Auto-approved drewstone PR — d095318d

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-08-01T22:48:29Z}

[tangletools]

✅ No Blockers — d095318d

Review health 100/100 · Reviewer score 76/100 · Confidence 65/100 · 7 findings (1 medium, 6 low)

	glm	deepseek	deepseek-flash	aggregate
Readiness	89	92	76	76
Confidence	65	65	65	65
Correctness	89	92	76	76
Security	89	92	76	76
Testing	89	92	76	76
Architecture	89	92	76	76

Reviewer score is advisory once the run is complete and the verdict has no blockers.

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

🟠 MEDIUM Check rename breaks out-of-tree consumers and invalidates persisted comparisons — src/contract/measured-comparison.ts

candidate-task-pass is renamed to no-task-regression with changed semantics. The comment argues a consumer now sees a MISSING check, but a consumer using checks.find(...)?.passed gets undefined (falsy => treated as failed) and one using !check.passed throws — two different, both-wrong failure modes depending on consumer style. More importantly, decision (reasons + contributingChecks) is inside agentImprovementMeasuredComparisonSchema, so verifyCandidateExperimentComparison (line 726) recomputes the content-addressed digest: any comparison persisted under the old code now FAILS digest verification. The source comment states 'No live run of th

🟡 LOW Test docblock still carries the retracted 'was held' framing — src/contract/measured-comparison.test.ts

The docblock at 1166-1171 says 'a candidate that lifted the paired mean by +0.53 with a bootstrap interval of [0.36, 0.65] was held with "candidate failed 2 benchmark tasks"', which reads as an observed live event. Commit d095318 explicitly retracted exactly this claim as unsourced ('no live run of this gate is on disk, and no arm of the work... ever called a model') and corrected only the source comment in measured-comparison.ts. The test's own fixture repairedNotRegressed(8) is the real, constructed case and the interval [0.36,0.65] is not reproduced anywhere. Fix: align the docblock with the retraction — say 'would be held' and drop the unsourced interval.

🟡 LOW Accounting line comment claims 'gate already holds' but line also appears on need_more_work — src/contract/measured-comparison.ts

The comment at line 529 says the accounting line is 'emitted only when the gate already holds'. The actual condition is !(shipped || sharedTaskFailures.length === 0), which triggers on BOTH 'hold' AND 'need_more_work' outcomes (any non-ship with shared failures). Not a functional bug — the line is correctly informational — but the comment is imprecise. Fix: change to 'emitted only when the gate did not ship'.

🟡 LOW Accounting line counts passed without accounting for completed — src/contract/measured-comparison.ts

repaired/shared/regressed buckets and the 'benchmark tasks: X repaired...' line are computed solely from adapter.passed, which is benchmarkResult.material.passed on the real path (line 1235), independent of completed. A timed-out candidate cell whose grader recorded passed=true is counted as 'repaired' (or excluded from 'still failing'), and one graded passed=false is counted as 'regressed' — so the operational text can mislabel an execution failure as a repair or regression. No false ship results (all-runs-completed gates independently), so this is a reporting-accuracy nit. Fix: bucket by completed first, or label the line as 'by graded

🟡 LOW Exact-sign reason surfaces a misleading 'Below 6 pairs' threshold — src/contract/measured-comparison.ts

The new last branch appends decision.methodDetail (from paired-promotion-decision.ts:287-291), which reads 'Below 6 pairs the interval is descriptive only; the decision is the exact one-sided sign test'. minimumPairs is max(requested, ceil(log2(1/0.025)))=6, but the exact-sign switch actually fires at BOOTSTRAP_GATE_MIN_N=20 (statistics.ts:1212). So for n in [6,19] the emitted reason states the interval is descriptive below 6 pairs — false for the current n, and wrong about the real cutoff. Pre-existing string, but this PR is what puts it into the public reasons. Fix: emit Below ${BOOTSTRAP_GATE_MIN_N} pairs in the methodDetail, or omit methodDetail here.

🟡 LOW McNemar p-value in reason string is unformatted raw float — src/contract/measured-comparison.ts

The McNemar reason interpolates p=${decision.mcnemar.pValue} with no formatting. For small discordant counts this is clean (e.g. p=0.5), but for larger samples the exact-test p-value can be an irrational float producing ugly output like p=0.3333333333333333. The decidePairedPromotion module formats its own p-values via fmt(x) (toFixed(4)). Consider applying the same formatting here for consistency.

🟡 LOW trailing dot on empty methodDetail produces orphan period in significance reason — src/contract/measured-comparison.ts

When decision.methodDetail is '' (the score-interval path), the reason string ends with '...did not clear 0.' — functionally harmless since the period just terminates the sentence, but it's a trailing-concatenation smell. The exact-sign path ('...did not clear 0. Below...') reads correctly. Not worth holding merge.

---

_{tangletools · 2026-08-01T23:32:34Z · trace}

[tangletools]

✅ Approved — 7 non-blocking findings — d095318d

Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 2 changed files. Global verifier still owns final merge decision.

Full immutable report for this review: trace

Summary comment for this run: full summary

---

_{tangletools · 2026-08-01T23:32:34Z · immutable trace}

[tangletools]

✅ Auto-approved drewstone PR — d061ceee

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-08-01T23:48:08Z}

[tangletools]

🟢 Value Audit — sound

Verdict	sound
Concerns	0 (none)
Heuristic	0.0s
Duplication	0.0s
Interrogation	94.2s (2 bridge agents)
Total	94.2s

💰 Value — sound

Replaces an unreachable perfection bar with a no-regression bar and fixes a self-contradicting significance reason; correct, in-grain, ships.

• What it does: Two fixes in evaluatePairedMeasurements (src/contract/measured-comparison.ts). (1) The task gate used to block on !candidate.passed for ANY cell — i.e. the candidate had to pass every benchmark task. It now blocks only when a candidate fails a cell the BASELINE passed (!candidate.passed && baseline.passed), renamed candidate-task-pass -> no-task-regression, and tracks a three-way split (re
• Goals it achieves: Make the promotion gate ship real improvements instead of rejecting winners it cannot distinguish from nulls, and make its refusals legible. (1) On any benchmark hard enough to be worth running nothing passes every task, so the old bar was unreachable and any improvement measured through it was unreadable — a null could just be a gate artifact. (2) An operator reading a refusal needs the reason to
• Assessment: Good change on its merits, built in the grain of the codebase. The predicate is visibly correct: improvement-without-regression is the honest bar, the improvement half is already carried by paired-significance and the quality half by critical-dimensions, and the per-task regression is the one piece that was missing — computed from adapter.passed projections already available for both arms, so it
• Better / existing approach: none — this is the right approach. I checked for an existing per-task regression primitive to reuse (grep'd regress across src; the closest kin is decidePairedPromotion / McNemar in src/paired-promotion-decision.ts and the continuous critical-dimensions check at measured-comparison.ts:381). Routing the binary task-pass outcome through that statistical machinery would be semantically wrong here:
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 2
• Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error

🎯 Usefulness — sound

Fixes an unreachable promotion bar (candidate had to pass every task) and a self-contradicting significance reason; both touch the central paired gate that two real publishers consume, and both reuse existing adapter/decision surface rather than inventing new machinery.

• Integration: evaluatePairedMeasurements (measured-comparison.ts:243) is reached by measuredComparisonFromCandidateExperiment (measured-comparison.ts:660) and measuredComparisonFromAgentProfileImprovementExperiment (profile-measured-comparison.ts:206); both feed decision.contributingChecks and decision.reasons into the published AgentImprovementMeasuredComparison / AgentProfileImprovementMeasuredComparison sche
• Fit with existing patterns: Follows the established grain: D1 reuses adapter.passed already projected for the baseline arm (projectRun enforces boolean), splits failures into repaired/regressed/shared consistent with the existing accounting style, and routes into the existing checks/hardFailure/reasons machinery; D2 reads fields off the existing PairedPromotionDecision object that decidePairedPromotion already produced, rath
• Real-world viability: Robust on edge inputs: projectRun throws if passed/completed are non-boolean so the regression predicate cannot see undefined; an incomplete candidate run is still caught independently by all-runs-completed and counted in hardFailure; regressedTasks.length>0 is in the hardFailure disjunction so a regression yields hold (not need_more_work), and the non-blocking accounting line is gated on !shipped
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260802T000357Z}

[tangletools]

⚠️ Review Interrupted — d061ceee

The review runner stopped before publishing a final verdict: webhook_restarted.

State	Detail
Interrupted	webhook restarted

No review verdict was produced for this run. Trigger a fresh review on the current PR head if the PR is still open.

_{tangletools · #518 · model: kimi-for-coding · updated 2026-08-02T00:06:10Z}

[tangletools]

✅ Auto-approved drewstone PR — d061ceee

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-08-02T00:07:27Z}

[tangletools]

🟢 Value Audit — sound

Verdict	sound
Concerns	0 (none)
Heuristic	0.0s
Duplication	0.0s
Interrogation	72.7s (2 bridge agents)
Total	72.7s

💰 Value — sound

Fixes two real defects in the paired promotion gate: (1) the task check required the candidate to pass EVERY task (unreachable on any hard benchmark) — now blocks only on per-task regressions; (2) the refusal reason reported a DIAGNOSTIC bootstrap bound instead of the interval that actually decided,

• What it does: Two behavior changes in evaluatePairedMeasurements. (1) Renames the candidate-task-pass check to no-task-regression and changes its predicate from '!candidate.passed on any cell' (perfection) to 'candidate failed a cell the baseline PASSED' (regression). A candidate that repairs some tasks while leaving shared failures failing now ships; a candidate that breaks a task the baseline solved still
• Goals it achieves: Make the promotion gate (a) reachable on realistic benchmarks — so an improvement is no longer reported as a defect merely because the candidate didn't solve every task the baseline also couldn't, and (b) honest in its refusal reasons — so an operator reading a hold sees which interval decided and why, not a number from a different estimator that never entered the decision. Net effect: the gate sh
• Assessment: Good change on its merits. D1 is a correct bar correction: 'improvement without regression' is the defensible promotion contract, the improvement half is already carried by paired-significance and the quality half by critical-dimensions, and the per-task regression the old check conflated with 'passed' was available from the same adapter.passed already projected for the baseline arm — no new evide
• Better / existing approach: none — this is the right approach. Searched for (a) existing per-task regression logic that could be reused: the critical-dimensions check (measured-comparison.ts:381-403) guards DIMENSION-score regressions via decidePairedPromotion, a different granularity from the task pass/fail flag, so not a duplicate — both are warranted. (b) An existing significance-reason helper: none existed; the old strin
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 2
• Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error

🎯 Usefulness — sound

Corrects the promotion gate's unreachable perfection bar (candidate had to pass every task) to 'improvement without regression' and rewrites a self-contradicting significance reason to name the interval that actually decided — both on the live paired-evaluation path used by every measured comparison

• Integration: evaluatePairedMeasurements (measured-comparison.ts:243) is the single paired evaluator invoked by both production consumers: measuredComparisonFromCandidateExperiment (measured-comparison.ts:660) and profileMeasuredComparisonFromCandidateExperiment (profile-measured-comparison.ts:206). Its output flows verbatim into decision.contributingChecks / decision.reasons on the schema-published AgentImprov
• Fit with existing patterns: Follows the established grain. The new regression check reuses the already-projected adapter.passed field — both arms already flow through projectRun (measured-comparison.ts:1098) which reads adapter.passed, so no new adapter surface, no new evidence path. The significanceReason rewrite now reads decision.low/high/statistic/indeterminate/mcnemar, which the statistics module documents as 'the inter
• Real-world viability: Holds across the paths that matter. The three-way task split reads booleans only (no numeric edge cases), and incomplete runs are still caught by the separate all-runs-completed check and counted in hardFailure (measured-comparison.ts:505-509), so a non-completing candidate cannot slip through on a passed=true fluke. significanceReason covers all four branches — fewRuns, degenerate/indeterminate,
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

No concerns — sound change, no better or existing approach found. ✅

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260802T000856Z}