feat(analyst): intent-divergence kind, cost-weighted failure-mode, knowledge capture

[drewstone]

Driven by reading a real session report, not a benchmark. On that report 7 of 13 findings restated counts the free deterministic pass already emits, and the causal findings that actually mattered were connected to nothing durable.

1. intent-divergence — a new kind, and the only one with free ground truth

Every other analyst guesses at quality. This one has an observed, labelled signal already sitting in every session: a corrective human turn ("no, I meant…", "stop", "why are you still asking") is a divergence event with a timestamp.

So the finding is not "the user corrected the agent" — that is counting, and the deterministic pass does it free. The kind walks backward from each correction to the earliest assistant turn already off-intent, names the signal that turn contradicted, and prices the gap as K burned turns. A finding without an anchor turn and a K is dropped by construction.

Second-order case handled separately: when the user clarifies rather than corrects, the agent's reading was defensible and the request was underspecified. Those charge a scaffolding:* locus — the fix is one sharp question before acting — not the agent's instructions.

A defect worth calling out, found by the adversarial reviewer and fixed here: every discovery pattern now opens with (?i). compileSearchRegex (src/trace-analyst/store-bounds.ts) adds CASE_INSENSITIVE only when a pattern starts with that flag, and corrections are overwhelmingly sentence-initial and capitalised — so the original lowercase patterns returned zero hits while looking completely correct. Verified by executing the store's own compile path against real corrections: 5/6 patterns match, where capitalised inputs previously matched none.

2. failure-mode — ranked by cost, not count

It reported "Bash is the dominant failure surface (54 error spans)" and "79/5260 tool calls errored". True, free, and useless: the deterministic analyzer emits exactly that.

An error the agent recovered from next turn cost ~nothing. One that ran twenty turns, or escaped to a human, cost real money. Findings now require a recovery boundary and the two spans that bound it, severity follows measured cost rather than frequency, and pure count aggregation is explicitly forbidden as duplicate output.

3. knowledge-capture — the missing wire

The knowledge-gap analyst correctly caught that the agent learned mid-session about a \ signup credit after filing a public issue with the wrong root cause. It learned, corrected itself, and the knowledge died at session end. We have a detector and a store and nothing joining them.

Pure transform: knowledge-gap findings whose subject routes to agent-knowledge:wiki:* become candidate pages carrying the claim, the recommended action, and the evidence spans as anchors. No I/O, no network, and a finding without evidence anchors is dropped rather than emitted with empty anchors. Candidates only — auto-writing model claims into a curated KB is precisely what the knowledge-poisoning analyst exists to detect.

Verification

tsc 0 · biome clean · 496 passed / 1 skipped across 35 files. finding-subject.test now derives its kind roster from DEFAULT_TRACE_ANALYST_KINDS instead of hardcoding four names, so the next kind added does not produce a spurious failure.

Each build was reviewed by an independent adversarial agent instructed to refute it and re-run the gates itself. Their unresolved findings are recorded below and are not fixed in this PR — I am not claiming they are clean.

Known-open, honestly

• knowledge-capture is not yet safe to wire to a write path. The reviewer executed the real agent-knowledge code against this module's output and falsified the claim that a reviewed candidate can be handed to the page store unchanged. The transform itself is correct and side-effect-free; the bridge claim is not. Treat the output as review material only.
• minimumEvidenceCitations: 2 does not enforce the pair it should. The factory counts distinct URIs, so two assistant spans satisfy it without ever quoting the corrective human turn. Enforcing "one human + one assistant" needs a postProcess reading role attributes, whose key differs across datasets.
• K lives in prose, not a typed field. Downstream ranking by burned-turn cost would have to regex it out of the claim string; a numeric field in the finding schema is the real fix.
• No analyst here has been run against a live trace store. These tests assert prompt structure and spec shape — that the protocol is written down, not that a model executes it. The honest next step is one scored run against sessions with hand-labelled corrective turns.

[tangletools]

✅ Auto-approved drewstone PR — b48b3074

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-31T23:08:02Z}

[tangletools]

🟡 Value Audit — sound-with-nits

Verdict	sound-with-nits
Concerns	1 (1 medium-concern)
Heuristic	0.0s
Duplication	0.1s
Interrogation	123.1s (2 bridge agents)
Total	123.2s

💰 Value — sound

Reframes failure-mode to price cost (not re-count free errors), adds an intent-divergence kind that exploits free ground truth (corrective human turns), and adds a pure knowledge-capture transform — all in the codebase's grain with no duplication.

• What it does: Three coupled changes to the analyst suite. (1) A NEW kind intent-divergence (src/analyst/kinds/intent-divergence.ts:63) that treats a corrective human turn as a labelled divergence event and walks BACKWARD to the earliest assistant turn already off-intent, pricing the gap as K burned turns; ambiguous-request cases route to scaffolding:* instead of the agent. (2) A reframe of failure-mode (s
• Goals it achieves: Stop the analyst from restating what the deterministic pass already emits for free. The PR body's claim that 7/13 findings on a real report duplicated free counts is directly addressed: failure-mode now forbids error-rate/signature/tool-count output and requires a measured cost instead; intent-divergence forbids bare correction-counting and requires an anchor turn + K. The second goal is durabilit
• Assessment: Good change, well-executed. Each piece follows the established kind grain exactly: same TraceAnalystDefinition shape, findingSubjectGrammarPromptFor(...), a KIND_EXPECTED_SUBJECTS entry, minimumEvidenceCitations gate, and matching test coverage (src/analyst/kinds/kinds.test.ts:182-236). The intent-divergence locus set reuses EXISTING FindingSubject variants (no new grammar variant, no ne
• Better / existing approach: none — this is the right approach. Searched for a pre-existing knowledge-capture or candidate-page builder (grep captureKnowledgeCandidates|KnowledgeCandidate|toWikiPage across src/) — none exists; this is net-new and correctly placed as a pure transform in the substrate. Searched for an existing divergence/intent kind — the closest is knowledge-gap's 'clarifying question' signal, but it does not
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 2
• Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound-with-nits

intent-divergence and the failure-mode reframe are well-built and wired into the default registry; knowledge-capture is a complete pure transform that is structurally unreachable because it was never exported from the package surface.

• Integration: intent-divergence is registered in DEFAULT_TRACE_ANALYST_KINDS (src/analyst/kinds/index.ts:45) and reaches the default registry via src/analyst/default-registry.ts:34 — any operator running the default suite gets it automatically. failure-mode is the same spec object, same path. KIND_EXPECTED_SUBJECTS now derives its roster from DEFAULT_TRACE_ANALYST_KINDS (finding-subject.test.ts:307-310), so the
• Fit with existing patterns: Follows the established kind-factory pattern exactly: TraceAnalystDefinition with minimumEvidenceCitations/toolGroup/limits mirrors knowledge-poisoning (src/analyst/kinds/knowledge-poisoning.ts:59). The minimumEvidenceCitations:2 contract is enforced structurally at src/analyst/kind-factory.ts:239, not just by prompt. The failure-mode reframe (price by recovery distance, forbid restating overview
• Real-world viability: The (?i) prefix fix is the load-bearing robustness claim and it is verified: compileSearchRegex at src/trace-analyst/store-bounds.ts:400-408 adds CASE_INSENSITIVE only when the pattern starts with (?i), so the lowercase correction patterns would silently match nothing against capitalised sentence-initial corrections without it. The prompt makes the flag mandatory and explains why (intent-divergenc
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

🎯 Usefulness Audit

🟠 knowledge-capture is not exported — its natural caller cannot reach it [integration] ``

captureKnowledgeCandidates and all KnowledgeCandidate* types appear only in src/analyst/knowledge-capture.ts and its test; they are absent from src/analyst/index.ts (which exports the other four KIND_SPECs at lines 264-270) and from src/index.ts. The module's docstring (knowledge-capture.ts:6-11) states its purpose is to join the knowledge-gap analyst to the @tangle-network/agent-knowledge store, and per CLAUDE.md's layering rule that join must live in the downstream package (agent-knowledge imp

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260731T231031Z}

[tangletools]

✅ No Blockers — b48b3074

Review health 100/100 · Reviewer score 69/100 · Confidence 65/100 · 9 findings (2 medium, 7 low)

	glm	deepseek	deepseek-flash	aggregate
Readiness	86	89	69	69
Confidence	65	65	65	65
Correctness	86	89	69	69
Security	86	89	69	69
Testing	86	89	69	69
Architecture	86	89	69	69

Reviewer score is advisory once the run is complete and the verdict has no blockers.

Full multi-shot audit completed 1/1 planned shots over 9 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 9 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 9 changed files. Global verifier still owns final merge decision.

🟠 MEDIUM INTENT_DIVERGENCE_KIND_SPEC is omitted from both public barrels while sibling specs are exported — src/analyst/kinds/index.ts

The new kind spec is exported from the internal kinds barrel (kinds/index.ts) and is present in DEFAULT_TRACE_ANALYST_KINDS, but neither public barrel re-exports it by name: src/index.ts lines 112-121 and src/analyst/index.ts lines 264-270 each export FAILURE_MODE_KIND_SPEC, IMPROVEMENT_KIND_SPEC, KNOWLEDGE_GAP_KIND_SPEC, KNOWLEDGE_POISONING_KIND_SPEC — but not INTENT_DIVERGENCE_KIND_SPEC. A consumer building a custom registry (e.g. failure-mode + intent-divergence without the full default suite) cannot import the new kind and has n

🟠 MEDIUM knowledge-capture module is unreachable: not exported from any public barrel and has no callers — src/analyst/knowledge-capture.ts

The PR's headline feature ('knowledge-gap findings become CANDIDATE wiki pages') is dead code from a consumer's perspective. captureKnowledgeCandidates is imported only by knowledge-capture.test.ts. It is not re-exported from src/analyst/index.ts or src/index.ts, and package.json's exports map exposes only './analyst' (dist/analyst/index.js) with no deep './analyst/knowledge-capture' entry, so the package's exports map blocks a deep import too. The transform is pure and tested, but nothing — no adapter, no registry hook, no gated activation path — invokes it, so the documented 'join' between the knowledge-gap analyst and the agent-knowledge store cannot be made by any consumer of this package. This is exactly the 'routing table rot' failure mode the module's own docstring warns about, appl

🟡 LOW Benchmark implementation digest bumped for finding-subject.ts only; behavior-defining kind sources stay unbound — src/analyst/benchmark-implementation.ts

The SHA bump in this PR is attributable entirely to src/analyst/finding-subject.ts — the only changed file in ANALYST_BENCHMARK_IMPLEMENTATION_FILES (verified by intersecting the PR's changed-file list with the base's manifest). Yet the behavior-defining changes this PR makes are outside the manifest: the failure-mode actor prompt was rewritten (version 1.2.0 -> 2.0.0, entirely new cost-bounding protocol), a brand-new analyst kind (intent-divergence) was added to DEFAULT_TRACE_ANALYST_KINDS, and knowledge-capture.ts was added — none of kinds/failure-mode.ts, kinds/intent-divergence.ts, kinds/index.ts, or knowledge-capture.ts are in the manifest, and the checker's transitive closure (from benchmark-command.ts) does not reach them. Because buildDefaultAnalystRegistry() now runs a different a

🟡 LOW new source files not in ANALYST_BENCHMARK_IMPLEMENTATION_FILES — src/analyst/benchmark-implementation.ts

src/analyst/knowledge-capture.ts is a new implementation file added in this PR but is not listed in ANALYST_BENCHMARK_IMPLEMENTATION_FILES. The individual kind files under src/analyst/kinds/ are also absent from this list (consistent: no kind file is tracked there). However, knowledge-capture.ts is not a kind — it is an implementation transform that converts findings to candidate pages, much like src/analyst/adapters.ts (also not tracked). If the benchmark intentionally excludes the knowledge-capture pipeline from its implementation fingerprint, this is fine. But if it should be part of the verifiable implementation surface, the SHA would under-report the changes. The comment at [lines 15-21](https://github.com/tangle-network/agent-eval/blob/b48b30747b3df65a7b2dc83467ef738947c78b

🟡 LOW candidate.heading stores kebab form while page.text renders humanized form — src/analyst/knowledge-capture.ts

When a locus carries a heading (e.g. subject = 'agent-knowledge:wiki:foo#eligibility'), candidate.heading = 'eligibility' (raw kebab from the parser) but page.text starts with '## Eligibility' (humanize(heading) in renderBody, line 280). A caller that tries to locate the rendered section by string-matching candidate.heading against page.text will fail — it must humanize first. The identity field and the display string use different conventions. Not a bug in the current code (heading is the section identity, not a display token), but a trap for the first integrator. Consider either storing the humanized form in a separate display field or documenting the transf

🟡 LOW knowledge-capture accepts all analystIds (caller must filter poisoning findings) — src/analyst/knowledge-capture.ts

The captureKnowledgeCandidates() function accepts any AnalystFinding whose subject parses as knowledge.wiki, regardless of analyst_id. Knowledge-poisoning findings (analyst_id='knowledge-poisoning') also emit agent-knowledge:wiki:* loci but describe pages that are WRONG. The doc at lines 29-33 states this is caller responsibility, and analystId is recorded on every candidate for downstream filtering. However, a caller that forgets to filter by analystId will silently import wrong knowledge. Consider either: (a) adding an optional analystIdFilter parameter, or (b) rejecting knowledge-poisoning findings with an explicit drop reason rather th

🟡 LOW knowledge-capture module ships with no production caller — src/analyst/knowledge-capture.ts

captureKnowledgeCandidates is exported but imported only by its own test file (grep confirmed: no caller in src/ outside knowledge-capture.test.ts). The module comment frames it as 'the join' between the knowledge-gap detector and the agent-knowledge store, but the actual join — a caller that feeds analyst findings in and hands candidates to the KB — is not in this PR. This is a defensible incremental-ship strategy (pure library, integration later), and the module docs explicitly state the caller-responsibility boundary. Flagging only so the global verifier knows the closed loop (detect → capture → write-gated) is not yet wired end-to-end; nothing breaks, but the user-visible gap the module claims to close is still open until an integrator ships.

🟡 LOW renderBody blockquote rendering can be broken by excerpt structure — src/analyst/knowledge-capture.ts

Excerpt lines are rendered as ' > ${l}' per line. An excerpt containing a blank line breaks the blockquote continuation (CommonMark requires '> ' on every line to stay in the block). An excerpt line that is just '---' renders as a horizontal rule inside the blockquote. This is by design ('the excerpt must survive unedited') and these are CANDIDATE pages for human review, never auto-written to the KB, so the blast radius is cosmetic markdown on a review artifact. No correctness impact on routing, grounding, or provenance.

🟡 LOW renderBody embeds claim/rationale/excerpt verbatim into markdown — structure injection into the candidate page — src/analyst/knowledge-capture.ts

The claim, rationale, and recommended_action are spliced into the body unescaped (line 280-283) and excerpts are blockquoted line-by-line (line 290). A claim that begins with '#', '##', '-', '>', or a backtick changes the rendered page structure, and a URI containing a backtick breaks the - code span`` (line 287). Impact is cosmetic and the page is reviewer-gated, but the KB linter/round-trip i

---

_{tangletools · 2026-07-31T23:13:04Z · trace}

[tangletools]

✅ Approved — 9 non-blocking findings — b48b3074

Full multi-shot audit completed 1/1 planned shots over 9 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 9 changed files. Global verifier still owns final merge decision. | Full multi-shot audit completed 1/1 planned shots over 9 changed files. Global verifier still owns final merge decision.

Full immutable report for this review: trace

Summary comment for this run: full summary

---

_{tangletools · 2026-07-31T23:13:04Z · immutable trace}