Skip to content

refactor: no harness is special — delete the name-keyed branches - #691

Merged
drewstone merged 2 commits into
mainfrom
refactor/no-special-harnesses
Aug 1, 2026
Merged

refactor: no harness is special — delete the name-keyed branches#691
drewstone merged 2 commits into
mainfrom
refactor/no-special-harnesses

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Continues #688 (piExecutor deleted) one layer down: logic that branches on a harness NAME rather than a declared capability.

Deleted — 7 sites

The name alias. materializerHarness() existed only to translate 'claude''claude-code', because local-harness.ts declared a private 3-member LocalHarness union shadowing the shared 13-member HarnessType. It used the executable name as the harness name; everywhere else in the stack correctly separates them (command: 'claude', harness 'claude-code'). Git blame confirms it was papered over the day someone first needed to pass a LocalHarness to the materializer. Fixing the union deletes the alias for free and stops it being re-invented at the next call site.

Three inert callers. dangerouslySkipPermissions: harness === 'claude' at three sites, all cargo-culting the consumer's own gate. A caller passing true for codex got identical behaviour to passing false — the flag was dropped silently. The comments at all three sites state the real intent, and it has nothing to do with a CLI name: "the candidate worktree is isolated and must be editable without an interactive permission prompt." That is a property of the workspace.

A dead switch case in cli-bridge (case 'claude' unreachable, since the only construction site always passes harness: 'claude-code').

A divergent duplicate list. shouldApplyHostAdmission hand-rolled the set of backends that never spawn a host CLI, while config.ts already declared it as data — and the two disagreed: nanoclaw was in the declared set, missing from the inline copy. A nanoclaw request consumed a host process slot it never needed and could take 503 admission_rejected / queue_full waiting for a subprocess it never spawns.

The security defect this refactor introduced, and how it was caught

Making the three callers unconditional turned codex's bypass argv on for every worktree worker. That argv was --dangerously-bypass-approvals-and-sandbox — codex's own help: "Skip all confirmation prompts and execute commands without sandboxing. EXTREMELY DANGEROUS."

Every codex worktree worker would have run with no OS sandbox. Writes stop being confined to the worktree, so a prompt-injected worker reaches ~/.ssh, ~/.aws, and secrets on the box. The "the worktree is disposable" reasoning is exactly backwards: the sandbox is what made the blast radius the worktree.

The stated motivation was false too. codex exec has no approval gate to stall on-a/--ask-for-approval exists only on the top-level codex, not the non-interactive exec subcommand. The flag bought nothing on the approvals axis and paid the entire sandbox for it.

Fixed: codex contributes --sandbox workspace-write -c approval_policy="never" — same non-interactive editing, writes still confined. The repo already proves that form works in CODEX_REPRODUCIBLE_ARGS. A test pins that codex's argv never contains the sandbox-surrendering flag, the option doc now states that a bypass never surrenders an OS sandbox, and the changelog line that claimed the opposite is corrected.

This is the failure mode worth naming: removing a special case can silently WIDEN a capability. Deleting harness === 'claude' did not just stop excluding codex — it started including it, in a privilege the author never intended to grant.

Gates

tsc 0 · biome clean (537 files) · 2246 passed / 6 skipped across 198 files · docs in sync.

`LocalHarness` was a private three-member vocabulary that spelled one
harness differently from the rest of the stack. It is now a narrowing of
the shared `HarnessType`, so `'claude'` becomes `'claude-code'` and
`materializerHarness()` — the alias that translated between the two —
is deleted. `claude` stays the executable name, in the table's `command`
field, reachable via the new `localHarnessExecutable()`.

Eleven name branches become rows on a table:

- Reasoning effort: `runWorktreeHarness` hard-refused any profile with
  `model.reasoningEffort` unless the harness was codex, and
  `harnessInvocation` silently dropped it. `HARNESS_INVOCATIONS` now
  carries a `reasoning` row per harness (`--effort` for claude-code,
  `--variant` for opencode, unchanged for codex), and the admission
  check reads the same rows so a refusal lands before any worktree
  exists.
- Permission bypass: `dangerouslySkipPermissions` was tested against
  `'claude'` in four places; three were caller-side duplication of the
  fourth, which dropped the flag for every other harness with no error.
  Each harness declares its own bypass argv; codex gets
  `--dangerously-bypass-approvals-and-sandbox` where it previously got
  nothing. Reproducible Codex argv is byte-identical — its controlled
  config already pins `approval_policy="never"` with the sandbox intact,
  so the blanket flag is suppressed rather than layered on top.
- System prompt: the four-arm `switch (plan.harness)` and its
  conflicting-argument guard collapse into one `HARNESS_SYSTEM_PROMPTS`
  row per harness. The guard's fail-open default for an unlisted harness
  is gone.
- `harness === 'cli-base'` was re-derived at three call sites; it is now
  `harnessRunsAgent` / `agentHarness` in `src/runtime/harness-role.ts`.

Deliberately kept, with the reason now written at each site: the
`codexReproducible && harness !== 'codex'` guards (a codex-specific
public option asserting caller self-consistency) and every
`ExecutorConfig.backend` switch (a union tag naming the materialization
contract, not a harness name).
Removing the per-harness conditional made all three callers pass
dangerouslySkipPermissions unconditionally, which turned codex's bypass
argv on for every worktree worker. That argv was
--dangerously-bypass-approvals-and-sandbox: codex's own help calls it
'Skip all confirmation prompts and execute commands without sandboxing.
EXTREMELY DANGEROUS.' Writes stopped being confined to the worktree, so a
prompt-injected worker could reach ~/.ssh, ~/.aws, and secrets on the box.

The stated motivation was also wrong: codex exec has no approval gate to
stall on — -a/--ask-for-approval exists only on the top-level codex, not
the non-interactive exec subcommand. The flag bought nothing on the
approvals axis and paid the entire sandbox for it.

Codex now contributes --sandbox workspace-write -c approval_policy="never":
same non-interactive editing, writes still confined. Same form the repo
already proves out in CODEX_REPRODUCIBLE_ARGS. The option doc now states
that a bypass never surrenders an OS sandbox, and a test pins that codex's
argv never contains the sandbox-surrendering flag.

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 1e5ed5e9

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-08-01T08:08:04Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Value Audit — sound

Verdict sound
Concerns 1 (1 weak-concern)
Heuristic 0.0s
Duplication 0.0s
Interrogation 198.4s (2 bridge agents)
Total 198.4s

💰 Value — sound

Replaces name-keyed harness branches with capability table rows, aligns the private LocalHarness spelling with the shared HarnessType, and fixes a real codex sandbox-surrender defect — coherent, in-grain, no better available approach.

  • What it does: Three concrete deltas, all in src/mcp/local-harness.ts:131-169, src/candidate-execution/system-prompt.ts:62-91, src/runtime/harness-role.ts:26-38. (1) LocalHarness narrows from a private 'claude'|'codex'|'opencode' vocabulary to Extract<HarnessType,'claude-code'|'codex'|'opencode'> (local-harness.ts:57), deleting the materializerHarness() alias that translated 'claude'→'claude-code' (was at origin
  • Goals it achieves: (1) Kill the name-as-capability antipattern so harness N+1 is one row, not N branch edits — verified the original HARNESS_INVOCATIONS already existed table-shaped, this completes it. (2) Close silent-drop: a caller requesting a capability the row omits is now refused up-front (worktree-harness.ts:486-506 + local-harness.ts:198-207 throw before any worktree exists), and system-prompt projection is
  • Assessment: Good change on its merits, built in the grain of the codebase. The repo already used table-driven per-harness rows (NATIVE_EXECUTABLES, the old HARNESS_INVOCATIONS, CODEX_REPRODUCIBLE_ARGS) and discriminated executable-vs-harness-name elsewhere; this just removes the one layer that violated it. The two behavior changes are well-motivated and well-executed: the codex sandbox fix is a genuine securi
  • Better / existing approach: none — this is the right approach. Checked node_modules/@tangle-network/agent-interface (0.40.0): it exports HarnessType (the 13-member union) but does NOT yet export harness-capabilities/harnessHonorsEffort/harnessReasoningCeiling predicates — so the local harness-role.ts copy is a justified bridge, not a reinvention, and the author flags it 'SHOULD UPSTREAM' at harness-role.ts:11. The native CLI
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 2
  • Bridge warning: opencode/kimi-for-coding/k2p7: opencode: opencode error

🎯 Usefulness — sound

Coherent table-driven refactor that aligns a private 3-member union with the shared HarnessType vocabulary, fixes two latent silent-drop bugs (codex bypass, reasoning effort), and is fully wired with no dead surface.

  • Integration: Every new export is consumed within this PR. DEFAULT_LOCAL_HARNESS → agentic-generator.ts:214, driver-loop-generator.ts:86, in-process-executor.ts:88. LOCAL_HARNESSES → bin-helpers.ts:90 (env var parsing). localHarnessExecutable → bench/src/swe-arena/proposer-provenance.mts:123. harnessSupportsReasoningEffort → worktree-harness.ts:498 (pre-flight admission). harnessRunsAgent → supervise.ts:265. ag
  • Fit with existing patterns: Excellent fit. The codebase already treats 'claude-code' as the canonical HarnessType id (sandbox-backend.ts:48, researcher.ts:192, delegates.ts:169, benchmark-report.test.ts) — LocalHarness was the lone outlier using the executable name 'claude'. This removes the one translation layer (materializerHarness) that existed only to bridge the two. The capability-row pattern (HARNESS_INVOCATIONS, HARNE
  • Real-world viability: Holds up off the happy path. The security-sensitive codex bypass was introduced broken in f344421 (--dangerously-bypass-approvals-and-sandbox, dropping the OS sandbox) and corrected in 1e5ed5e to '--sandbox workspace-write -c approval_policy="never"', keeping the sandbox; local-harness.test.ts:1397 explicitly asserts the dangerous flag is absent. Reasoning effort is fail-closed: claude-code refuse
  • Model: opencode/zai-coding-plan/glm-5.2
  • Bridge attempts: 1

💰 Value Audit

🟡 Reasoning-capability knowledge now split across two layers that can drift [maintenance] ``

harnessSupportsReasoningEffort (local-harness.ts:191-196) and its CLAUDE_CODE/OPENCODE/CODEX _REASONING_LEVELS tables (local-harness.ts:68-100) encode 'which reasoning levels this CLI can spell' locally, while HarnessType's own capability table in agent-interface (the future harnessHonorsEffort/harnessReasoningCeiling the harness-role.ts:11 comment names) encodes the canonical ceiling. Today they agree, but a level added upstream (or a CLI flag rename) would need two edits with no compile-time l


What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260801T081154Z

@drewstone
drewstone merged commit ad57c08 into main Aug 1, 2026
3 of 4 checks passed
@drewstone drewstone mentioned this pull request Aug 1, 2026
drewstone pushed a commit that referenced this pull request Aug 1, 2026
…tionale

#691 renamed LocalHarness 'claude' to 'claude-code' but left this bench
assertion expecting the old spelling, so `pnpm run verify:bench` has
failed on every commit since ad57c0872913a1 was the last green one.

This blocks the release, not just CI: publish.yml runs the same
`pnpm run verify:bench` in its verify job before packing, so a tag would
fail the gate and never publish.

Only the harness NAME moves. The bare 'claude' in quant-loop.mts and
swe-code-improve.mts is the EXECUTABLE name, which the rename
deliberately kept, so both are left alone.
drewstone added a commit that referenced this pull request Aug 1, 2026
* chore(release): 0.119.0

0.118.0 bumped package.json on main but was never tagged, so the pi
executor deletion (fee8898) never reached the registry — the published
latest is still 0.117.0, which ships piExecutor. Main has since taken a
second BREAKING change (#691, the LocalHarness 'claude' -> 'claude-code'
rename) that sat under an "Unreleased" changelog heading with no version
covering it.

Publishing main as 0.118.0 would ship that rename under a number whose
changelog does not mention it, so this cuts 0.119.0 instead: the
Unreleased section becomes 0.119.0, and the 0.118.0 section records that
it never published and that its contents ship here. The registry goes
0.117.0 -> 0.119.0.

Version-embedded artifacts regenerated with release:prepare, so the
fixture, primitive-catalog, and canonical-api gates stay green.

* fix(bench): assert the renamed claude-code harness in the proposer rationale

#691 renamed LocalHarness 'claude' to 'claude-code' but left this bench
assertion expecting the old spelling, so `pnpm run verify:bench` has
failed on every commit since ad57c0872913a1 was the last green one.

This blocks the release, not just CI: publish.yml runs the same
`pnpm run verify:bench` in its verify job before packing, so a tag would
fail the gate and never publish.

Only the harness NAME moves. The bare 'claude' in quant-loop.mts and
swe-code-improve.mts is the EXECUTABLE name, which the rename
deliberately kept, so both are left alone.

---------

Co-authored-by: Drew Stone <hello@tangle.tools>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants