Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,38 @@
# Changelog

## Unreleased

### No harness is special: eleven name branches become table rows

BREAKING. `LocalHarness` was a private three-member vocabulary (`'claude' | 'codex' | 'opencode'`) that spelled one harness differently from every other layer in the stack. It is now a narrowing of the shared `HarnessType`: **`'claude'` is renamed to `'claude-code'`**. `claude` remains the EXECUTABLE name and lives only in the harness table's `command` field.

Callers to update: `runLocalHarness({ harness })`, `harnessInvocation(harness, …)`, `runWorktreeHarness({ harness })`, `agenticGenerator({ harness })`, `driverLoopGenerator({ harness })`, `createInProcessExecutor({ harnesses })`, `AuthoredHarness.harness`, and the `AGENT_RUNTIME_LOCAL_HARNESSES` env list. Anything that passed `'claude'` passes `'claude-code'`; `codex` and `opencode` are unchanged.

Deleting the alias removed `materializerHarness()` outright — a `LocalHarness` is now handed straight to the profile materializer with no translation.

**Reasoning effort now reaches claude-code and opencode.** `runWorktreeHarness` used to hard-REFUSE any profile carrying `model.reasoningEffort` unless the harness was codex, and `harnessInvocation` silently dropped it. Both read one capability table now:

- `claude-code` → `--effort <low|medium|high|xhigh|max>`; canonical `ultracode` is native `max`.
- `opencode` → `--variant <variant>`; canonical `ultracode` is `max`.
- `codex` → `-c model_reasoning_effort="…"`, unchanged.

A level a harness genuinely cannot express is still refused, and the refusal now lands in the pre-flight admission check (before any worktree exists) because the guard and the argv builder read the same rows. claude-code refuses `none` and `minimal` (its `--effort` has no such level); opencode refuses `none` (thinking-off is the absence of the flag).

**Permission bypass is a property of the workspace, not of one CLI.** `dangerouslySkipPermissions` was tested against `'claude'` in four places; three were caller-side duplication of the fourth, which dropped the flag for every other harness with no error. Each harness now declares its own bypass argv:

- `claude-code` → `--dangerously-skip-permissions` (unchanged).
- `codex` → `--sandbox workspace-write -c approval_policy="never"`. NEW: a codex worker in a disposable worktree previously had its bypass request silently dropped. It edits non-interactively now and **keeps its OS sandbox** — writes stay confined to the workspace. `--dangerously-bypass-approvals-and-sandbox` is deliberately NOT used: `codex exec` has no approval gate to stall on (`-a/--ask-for-approval` exists only on the top-level `codex`), so it would surrender the sandbox for nothing, and the sandbox is what keeps a worker's blast radius equal to its worktree.
- `opencode` → nothing; `opencode run` has no approval gate.
- Reproducible Codex is unchanged: its controlled config already pins `approval_policy="never"` with the sandbox intact, so the blanket bypass flag is suppressed rather than layered on top. Reproducible argv is byte-identical to 0.118.0.

**Other name branches replaced by rows, with no behaviour change:**

- `projectCandidateSystemPrompt`'s four-arm `switch (plan.harness)` and its conflicting-argument guard are now one `HARNESS_SYSTEM_PROMPTS` row per harness (executable + projection + conflict predicate). The guard's fail-OPEN default for an unlisted harness is gone: no row means the projection is refused.
- `harness === 'cli-base'` was re-derived at three call sites; it is now `harnessRunsAgent` / `agentHarness` in `src/runtime/harness-role.ts`.
- New exports on `@tangle-network/agent-runtime/mcp`: `DEFAULT_LOCAL_HARNESS`, `LOCAL_HARNESSES`, `localHarnessExecutable`, `harnessSupportsReasoningEffort`.

Deliberately KEPT: the `codexReproducible && harness !== 'codex'` guards (a codex-specific public option asserting caller self-consistency, not behaviour varying by name), and every `ExecutorConfig.backend` switch (a discriminated-union tag naming the materialization contract, not a harness name). Both now say so at the site.

## 0.118.0

### pi runs through the bridge, like every other harness
Expand Down
2 changes: 1 addition & 1 deletion bench/gen3-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"repsPerInstance": 2,
"premeasuredBaselinePath": "/tmp/claude-1000/-home-drew-code-supervisor-lab/f06fd156-042a-4ef9-bd88-f2ec7f52b90c/scratchpad/hh/gen3/premeasured-baseline.json",
"maxShots": 3,
"proposerHarness": "claude",
"proposerHarness": "claude-code",
"proposerTimeoutMs": 2400000,
"analystModels": [
"glm-5.2",
Expand Down
2 changes: 1 addition & 1 deletion bench/gen4-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"repsPerInstance": 2,
"premeasuredBaselinePath": "/tmp/claude-1000/-home-drew-code-supervisor-lab/f06fd156-042a-4ef9-bd88-f2ec7f52b90c/scratchpad/hh/gen4/premeasured-baseline.json",
"maxShots": 3,
"proposerHarness": "claude",
"proposerHarness": "claude-code",
"proposerTimeoutMs": 2400000,
"analystModels": [
"glm-5.2",
Expand Down
2 changes: 1 addition & 1 deletion bench/gen5-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"repsPerInstance": 2,
"premeasuredBaselinePath": "/tmp/claude-1000/-home-drew-code-supervisor-lab/f06fd156-042a-4ef9-bd88-f2ec7f52b90c/scratchpad/hh/gen5/premeasured-baseline.json",
"maxShots": 3,
"proposerHarness": "claude",
"proposerHarness": "claude-code",
"proposerTimeoutMs": 2400000,
"analystModels": [
"glm-5.2",
Expand Down
6 changes: 3 additions & 3 deletions bench/src/quant-arena/quant-loop.mts
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,11 @@ export const PINNED_BASELINES: Record<string, GenerateSignals> = {
/** The two demo author seats: the plain author and the quant lens. */
export function defaultQuantProposers(): ProposerSpec[] {
return [
{ name: 'default-author', profile: 'default-author.profile.json', harness: 'claude' },
{ name: 'default-author', profile: 'default-author.profile.json', harness: 'claude-code' },
{
name: 'quant-researcher',
profile: join(QUANT_PROFILES_DIR, 'quant-researcher.profile.json'),
harness: 'claude',
harness: 'claude-code',
lens:
'Favor ONE economically-motivated effect (trend, mean reversion, vol targeting) with few parameters. ' +
'State the regime in which it should work and keep turnover low enough that 15bps a side cannot eat the edge.',
Expand Down Expand Up @@ -218,7 +218,7 @@ async function claudeShot(opts: {
const res = await run('claude', argv, {
stdin: opts.prompt,
cwd: opts.cwd,
env: proposerShotEnv('claude'),
env: proposerShotEnv('claude-code'),
timeoutMs: opts.timeoutMs,
})
if (res.code !== 0) {
Expand Down
4 changes: 2 additions & 2 deletions bench/src/rollout-ledger/backfill-swe-arena.test.mts
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ async function buildFixtureTree(): Promise<void> {
candidateIndex: 0,
shot: 1,
maxShots: 3,
harness: 'claude',
harness: 'claude-code',
model: null,
promptSha256: 'sha256:abc',
startedAt: '2026-07-22T19:27:18.350Z',
Expand Down Expand Up @@ -256,7 +256,7 @@ describe('backfillSweArena', () => {
expect(proposer?.messages).toHaveLength(2)
expect(proposer?.outcome.reward).toBe(1)
expect(proposer?.outcome.reward_source).toBe('swe-arena-official-judge/candidate-resolved-fraction')
expect(proposer?.policy).toMatchObject({ harness: 'claude', model: 'claude-fable-5' })
expect(proposer?.policy).toMatchObject({ harness: 'claude-code', model: 'claude-fable-5' })
expect(proposer?.cost.tokens_out).toBe(462)
expect(proposer?.task).toMatchObject({ suite: 'swe-arena-proposer', rep: 1 })

Expand Down
8 changes: 4 additions & 4 deletions bench/src/swe-arena/activation.test.mts
Original file line number Diff line number Diff line change
Expand Up @@ -245,9 +245,9 @@ describe('activation-predicate prefilter', () => {

it('kills a candidate without .improve/activation.json (stage activation-predicate) and passes one WITH it', async () => {
const proposers: ProposerSpec[] = [
{ name: 'with-predicate', harness: 'claude' },
{ name: 'without-predicate', harness: 'claude' },
{ name: 'invalid-predicate', harness: 'claude' },
{ name: 'with-predicate', harness: 'claude-code' },
{ name: 'without-predicate', harness: 'claude-code' },
{ name: 'invalid-predicate', harness: 'claude-code' },
]
const gen = fanOutLoopsGenerator(config(proposers), {
author: async (proposer, args) => {
Expand Down Expand Up @@ -284,7 +284,7 @@ describe('activation-predicate prefilter', () => {
})

it('does not require a predicate when the gate is off (gen-4 behavior unchanged)', async () => {
const cfg = config([{ name: 'legacy', harness: 'claude' }])
const cfg = config([{ name: 'legacy', harness: 'claude-code' }])
cfg.activationGate = false
const gen = fanOutLoopsGenerator(cfg, {
author: async (_p, args) => {
Expand Down
8 changes: 4 additions & 4 deletions bench/src/swe-arena/gepa-seat.test.mts
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ describe('validateGepaSeat', () => {
expect(() => validateGepaSeat(seat())).not.toThrow()
expect(() => validateGepaSeat(seat({ engine: 'omni', maxMetricCalls: 8 }))).not.toThrow()
expect(isGepaSeat(seat())).toBe(true)
expect(isGepaSeat({ name: 'x', harness: 'claude' })).toBe(false)
expect(isGepaSeat({ name: 'x', harness: 'claude-code' })).toBe(false)
})

it('requires a surface inside the declared change-space', () => {
Expand All @@ -68,7 +68,7 @@ describe('validateGepaSeat', () => {
})

it('rejects harness-seat fields on an engine seat instead of silently ignoring them', () => {
expect(() => validateGepaSeat(seat({ harness: 'claude' }))).toThrow(/'harness' belongs to harness-authored/)
expect(() => validateGepaSeat(seat({ harness: 'claude-code' }))).toThrow(/'harness' belongs to harness-authored/)
expect(() => validateGepaSeat(seat({ merge: true }))).toThrow(/'merge'/)
expect(() => validateGepaSeat(seat({ model: 'x' }))).toThrow(/'model'/)
expect(() => validateGepaSeat(seat({ profile: 'p.json' }))).toThrow(/'profile'/)
Expand Down Expand Up @@ -234,7 +234,7 @@ describe('captureProposerProvenance with a gepa seat', () => {
return { code: 0, stdout: 'source', stderr: '' }
}
it('records engine, surface, gepa version, bridge module, and the python runtime as harnessVersion', async () => {
const record = await captureProposerProvenance([{ name: 'claude-author', harness: 'claude' }, seat()], {
const record = await captureProposerProvenance([{ name: 'claude-author', harness: 'claude-code' }, seat()], {
exec: okExec,
readSettingsModel: () => 'settings-model',
})
Expand All @@ -251,7 +251,7 @@ describe('captureProposerProvenance with a gepa seat', () => {
expect(gepa.harness).toBeUndefined()
// The claude seat is untouched by the gepa capture path.
expect(record.proposers.find((p) => p.name === 'claude-author')).toMatchObject({
harness: 'claude',
harness: 'claude-code',
settingsModel: 'settings-model',
})
})
Expand Down
16 changes: 8 additions & 8 deletions bench/src/swe-arena/outer-loop.mts
Original file line number Diff line number Diff line change
Expand Up @@ -646,7 +646,7 @@ export interface OuterLoopConfig {
* budget.maxImprovementShots; the LIB owns the dial (capabilities.mts
* fails loud on a substrate that would drop it). */
maxShots: number
proposerHarness: 'claude' | 'codex' | 'opencode'
proposerHarness: 'claude-code' | 'codex' | 'opencode'
proposerTimeoutMs: number
/** GEN-3 proposer fan-out: N proposers author candidates CONCURRENTLY, each
* an AgentProfile-pinned harness invocation (see proposer-fanout.mts).
Expand Down Expand Up @@ -759,7 +759,7 @@ export function defaultRound4Config(
// bootstrap run writes it; the lib validates it on every consumption.
premeasuredBaselinePath: join(hh, 'r4', 'premeasured-baseline.json'),
maxShots: 3,
proposerHarness: 'claude',
proposerHarness: 'claude-code',
// Per author SHOT (agenticGenerator timeoutMs). 20 min timed out 3× under
// degraded capacity in gen-1 ("author shot timed out") — doubled to 40 min.
proposerTimeoutMs: 2_400_000,
Expand Down Expand Up @@ -858,18 +858,18 @@ export function defaultGen3Config(
const base = defaultRound4Config(hh, opts)
const outDirName = opts.outDirName ?? 'gen3'
const proposers: ProposerSpec[] = [
{ name: 'default-author', profile: 'default-author.profile.json', harness: 'claude' },
{ name: 'default-author', profile: 'default-author.profile.json', harness: 'claude-code' },
{
name: 'mechanics-author',
profile: 'default-author.profile.json',
harness: 'claude',
harness: 'claude-code',
diagnosisSlice: 'mechanics',
lens: 'Focus on MECHANICS: worker lifecycle, sandbox/clone contracts, settlement and delivery paths. Prefer code-path fixes over prompt wording.',
},
{
name: 'prompts-author',
profile: 'default-author.profile.json',
harness: 'claude',
harness: 'claude-code',
diagnosisSlice: 'prompts',
lens: 'Focus on PROMPTS: worker/brain instruction wording, placement guidance, self-check discipline. Prefer prompt/instruction changes over code-path rewrites.',
},
Expand Down Expand Up @@ -945,10 +945,10 @@ export function defaultGen4Config(
): OuterLoopConfig {
const base = defaultGen3Config(hh, { outDirName: opts.outDirName ?? 'gen4' })
const proposers: ProposerSpec[] = [
{ name: 'claude-author', profile: 'default-author.profile.json', harness: 'claude' },
{ name: 'claude-author', profile: 'default-author.profile.json', harness: 'claude-code' },
{ name: 'glm-author', harness: 'opencode', model: 'zai-coding-plan/glm-5.2' },
...(opts.includeCodex === false ? [] : [{ name: 'codex-author', harness: 'codex' } satisfies ProposerSpec]),
{ name: 'merge-author', profile: 'default-author.profile.json', harness: 'claude', merge: true },
{ name: 'merge-author', profile: 'default-author.profile.json', harness: 'claude-code', merge: true },
]
return {
...base,
Expand Down Expand Up @@ -1263,7 +1263,7 @@ const CODEX_AMBIENT_AUTH_VARS = ['OPENAI_API_KEY', 'OPENAI_BASE_URL'] as const

export function proposerShotEnv(harness: OuterLoopConfig['proposerHarness']): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = { ...process.env }
if (harness === 'claude') {
if (harness === 'claude-code') {
for (const name of CLAUDE_AMBIENT_AUTH_VARS) delete env[name]
}
if (harness === 'codex') {
Expand Down
4 changes: 2 additions & 2 deletions bench/src/swe-arena/proposer-fanout.mts
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ export interface ProposerSpec {
profile?: string
/** Required for harness-authored seats. Absent on an engine seat
* (`engine` set) — enforced both ways at generator construction. */
harness?: 'claude' | 'codex' | 'opencode'
harness?: 'claude-code' | 'codex' | 'opencode'
/** GEN-4 pinned model id, threaded to the harness CLI as `-m <model>` via
* the author profile's `model.default` (harnessInvocation maps it for all
* three harnesses). Unset = the CLI's own resolved model (its login/settings
Expand Down Expand Up @@ -201,7 +201,7 @@ export function loadAuthorProfile(spec: ProposerSpec): AgentProfile | undefined

/** The gen-2 author, codified: one bare-profile claude proposer. */
export function defaultProposers(): ProposerSpec[] {
return [{ name: 'default-author', profile: 'default-author.profile.json', harness: 'claude' }]
return [{ name: 'default-author', profile: 'default-author.profile.json', harness: 'claude-code' }]
}

/** The profile the author shot actually runs: the loaded profile (if any) with
Expand Down
Loading
Loading