Skip to content

[Air #22] feat: add opencode as a consult lane (Grok) - #24

Merged
pseudoseed merged 10 commits into
mainfrom
builder/air-22
Aug 21, 2026
Merged

[Air #22] feat: add opencode as a consult lane (Grok)#24
pseudoseed merged 10 commits into
mainfrom
builder/air-22

Conversation

@pseudoseed

@pseudoseed pseudoseed commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Summary

Adds opencode as a fifth consult lane, running Grok (xai/grok-4.6 by default). It is the only reviewer on an account none of the other lanes share, which is the entire point.

Closes #22

Which lanes reviewed this PR

Stated plainly, because the honest answer is also the argument for the change:

Lane Result
claude APPROVE (HIGH confidence)
opencode (xai/grok-4.6) COMMENT (HIGH confidence) — found a real defect, see below
codex QUOTA-BLOCKED — "You've hit your usage limit… try again at Aug 27th, 2026 4:01 PM"
gemini SKIPPEDagy exited 1; quota resets Aug 28

Two of three default lanes are down. That is not background colour, it is the PR's justification arriving on schedule: four merges shipped today with two of three reviewers, and #23 is currently held at one of three.

Note also what gemini's absence looked like from porch's side. The skip emits VERDICT: COMMENT, and allApprove counts COMMENT as an approval — so a lane that never ran registered as a lane that approved. That is #20, occurring live, inside the review of the PR that cites it.

The new lane reviewed its own PR. A lane approving the change that adds it is weak evidence standing alone, and a reader should be able to see that and discount it. What makes it worth reporting is that it did not approve — it found a real bug in its own implementation, in the exact failure class this PR exists to reduce.

The defect the new lane found in itself

The issue requires the lane to record which model produced a review (Grok 4.6 and Grok 4.3 are not interchangeable evidence). I did that with a provenance banner at the top of the review file. The banner is 76 characters.

parseVerdict treats output under 50 characters as REQUEST_CHANGES — a floor meant to catch a lane that produced nothing useful. Anything longer with no verdict line falls through to COMMENT, which allApprove counts as an approval. So the banner converted the safe default into the unsafe one:

parseVerdict('ok')          -> REQUEST_CHANGES
parseVerdict(header + 'ok') -> COMMENT

Verified directly against the built parser, not taken on the reviewer's word. The PR claiming to reduce #20's blast radius was opening a fresh #20 hole.

The header is not really the bug

That 50-character floor is a proxy — length standing in for "a review actually happened" — and it was never measuring the right thing. It held by luck. Any lane that prefixes provenance, a banner, a model id, or a timestamp defeats it, and my header did not break a working guard so much as expose one that was already wrong.

This matters for whoever touches it next: do not fix this by raising 50 to 100. That reintroduces the same hole with the next slightly longer header.

Ask the real question instead. findVerdict() is extracted from parseVerdict() and returns the verdict a review states, or null. parseVerdict structurally cannot express that — a stated VERDICT: COMMENT and no verdict at all return the same value — and parseVerdict keeps its exact prior behaviour, now written as findVerdict(output) ?? 'COMMENT'.

This is the same distinction #20 needs. Whoever picks up #20 should reuse findVerdict rather than write a second one.

The opencode lane then hard-fails a protocol-mode review that states no verdict: a reviewer which produced output but no verdict has not reviewed. General mode (--prompt) is exempt — a question is not a review, and none was asked for.

What Changed

opencode joins VALID_LANE_NAMES and MODEL_CONFIGURABLE_LANES, gets a MODEL_CONFIGS entry, and dispatches through a runner of its own. The role folds into the prompt because opencode has no system-prompt flag — following the hermes precedent, temp-file path included, rather than inventing a second approach.

Lane opencode
Command opencode run -m <id> <prompt>
Default model xai/grok-4.6
Configurable consult.models.opencode, --model-id
In default rotation No — available, opt-in

Key Decisions

1. The model id is validated before the spawn — an exception to the no-catalog rule, deliberately.

consult-lanes.ts opens by forbidding local catalogs of model ids, because a catalog goes stale the day a provider ships a model. This lane checks against opencode models anyway, and the distinction is that this is not a list Codev ships: it is the provider tool answering for itself at call time, which is the same authority the rule defers to. If the listing fails — or returns nothing provider/model-shaped — the check stands down and the provider is authoritative again.

It has to happen before the spawn because opencode's own rejection is unusable:

$ opencode run -m x-ai/grok-4.6 "say hi"
Error: {"name":"UnknownError","data":{"message":"Unexpected server error."}}
exit 1, empty stdout

That names neither the model nor the mistake. What the lane says instead:

$ consult -m opencode --model-id x-ai/grok-4.6 --prompt "hi"
Unknown opencode model "x-ai/grok-4.6" (from --model-id).
`opencode models` on this machine offers: ..., "xai/grok-4.6", ...
Did you mean "xai/grok-4.6"? The model name is right; the provider prefix is not.
Codev does not fall back to a default model — correct the id at the source above.
exit 1

2. The lane hard-fails; it never skips.

The gemini/agy lane degrades to a non-blocking skip because an unauthenticated agy is a routine state and wedging every phase on it would be worse than losing a lane. opencode has no equivalent fragility. Missing CLI, unknown id, non-zero exit, empty output, and (in protocol mode) a verdict-less review all throw — and each discards any stale review file first, so porch cannot read an earlier iteration's verdict as this one's.

3. A test-isolation guard, mirroring the agy lane's (cluesmith#1323).

resolveOpencodeBin() refuses to resolve the real binary under a test runner without a pinned CODEV_OPENCODE_BIN. Not incidental — an unpinned suite bills a real Grok call per spawn. The guard sits at resolution rather than at the spawn because the pre-flight executes the binary.

4. Defaults untouched.

porch.consultation.models still defaults to ["gemini", "codex", "claude"]. Changing the default rotation is an architectural call the issue does not make.

Test Plan

  • 56 tests across opencode-lane.test.ts (39) and parse-verdict.test.ts (17), using a real fake opencode binary so argv, exit codes and the stdout/stderr split are genuinely exercised rather than mocked
  • DEFAULT_OPENCODE_MODEL pinned in default-models.test.ts, the designated edit point for shipped defaults
  • Consult + porch-verdict + lib suites: 281 passing across 14 files
  • Full suite: 5533 passed, 1 failedsession-manager.test.ts > respects maxRestarts limit, a timeout-based test that failed while two vitest suites ran concurrently and passes standalone (91/91 in 28s). Unrelated to consult; not skipped, since a test that passes cleanly on its own should not be annotated away.
  • Build passes (porch check 22)
  • Verified end to end against the real CLI and real Grok, which the issue asks for explicitly

The end-to-end run:

$ consult -m opencode --prompt "…" --output /tmp/oc-e2e-review.md
[OPENCODE] model: xai/grok-4.6
[opencode completed in 36.3s]     exit 0

$ node -e "…parseVerdict(fs.readFileSync('/tmp/oc-e2e-review.md','utf-8'))"
parsed verdict: APPROVE

The issue's item 5 said "verdict parsing needs no change if the lane writes a VERDICT: line to stdout — confirm it does before assuming." The premise is confirmed: opencode writes its banner and tool trace to stderr and only assistant text to stdout, and porch's own parser reads the resulting file as APPROVE.

The conclusion did not survive, and this PR does change verdict parsing — see the defect above. parseVerdict's behaviour is unchanged; what is new is findVerdict, and the lane refusing a protocol-mode review that states no verdict. Flagged explicitly because item 5 predicted otherwise, and because verdict.ts is shared porch code that a reviewer scanning a consult-lane PR would not expect to see touched.

Review Notes

  • The catalog-validation exception (assertOpencodeModelAvailable) runs against the grain of consult-lanes.ts's opening comment. The argument is in the docstring; if the distinction between "a catalog Codev ships" and "the provider tool answering for itself" does not hold, this is the thing to push back on.
  • findVerdict is a change to shared porch code, not just to the new lane. parseVerdict's behaviour is unchanged and pinned by its existing tests plus six new ones.
  • protocol-schema.json gained hermes as well as opencode. The enum was missing hermes already — editor tooling only, validating nothing at runtime, but it red-squiggled a long-valid lane. Called out because it is beyond the issue.
  • Docs touched in both trees (codev/ and codev-skeleton/), all four byte-identical copies of the consult skill, and the CLAUDE.md/AGENTS.md pair.

codev doctor already probed for OpenCode as an AI CLI dependency — further evidence for the issue's read that the lane's absence was an oversight rather than a decision.

pseudoseed and others added 4 commits August 21, 2026 16:35
consult had four lanes and no reason for opencode not to be one. Adding it
gives reviews a third independent reviewer on an account none of the existing
lanes share — which is what today cost us: codex has been quota-exhausted since
before 08:00 UTC, and four merges shipped with two of three lanes.

The lane is `opencode run -m <id> <prompt>`, defaulting to `xai/grok-4.6`. Role
folds into the prompt (opencode has no system-prompt flag), following hermes
rather than inventing a second approach. Verdict parsing is untouched: stdout is
plain assistant text with the VERDICT line intact, live-probed rather than
assumed.

Two things it does differently from the lanes it sits beside:

**The model id is checked before the spawn.** `x-ai/grok-4.6` — the spelling most
other tooling uses — comes back from the provider as `UnknownError: Unexpected
server error` with empty stdout, naming neither the model nor the mistake. So the
id is checked against `opencode models`, and a miss says which prefix this machine
actually has. That is not the hardcoded catalog consult-lanes.ts forbids: the list
is the provider tool answering for itself at call time, and if the listing fails
the check stands down and the provider is the authority again.

**Nothing degrades into a passing review.** The agy lane skips non-blockingly
because an unauthenticated agy is a routine state; opencode has no such property,
and #20 is standing evidence that a lane producing nothing is counted as an
approval. Missing CLI, unknown id, non-zero exit and empty output all throw, and
all discard any stale review file first so porch cannot read an earlier
iteration's verdict as this one's.

The review names the model that wrote it — Grok 4.6 and Grok 4.3 are not
interchangeable evidence.

`porch.consultation.models` still defaults to gemini/codex/claude; opencode is
available, not conscripted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both trees, since a lane adopters can select is framework content: the command
reference and the consult skill (all four byte-identical copies), plus CLAUDE.md
and its AGENTS.md twin.

The `protocol-schema.json` lane enum was missing `hermes` as well as `opencode` —
editor tooling only, but it red-squiggled a lane that has been valid for a long
time, so it now lists everything VALID_LANE_NAMES accepts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pseudoseed and others added 6 commits August 21, 2026 17:03
Found by the opencode lane reviewing its own PR, on its first run, in the exact
failure class the lane exists to reduce.

`opencodeReviewHeader` — the provenance banner added to satisfy the issue's
"record which model the lane used" — is 76 characters. `parseVerdict` treats
output under 50 characters as REQUEST_CHANGES; anything longer with no verdict
line falls through to COMMENT, and `allApprove` counts COMMENT as an approval.
So the banner converted the safe default into the unsafe one for any short
review:

    parseVerdict('ok')          -> REQUEST_CHANGES
    parseVerdict(header + 'ok') -> COMMENT

The PR that claims to reduce #20's blast radius was opening a fresh #20 hole.

The header is not really the bug. That 50-character floor is a PROXY — length
standing in for "a review actually happened" — and it was never measuring the
right thing. It held by luck, and any lane that prefixes provenance, a banner, a
model id or a timestamp defeats it. Raising 50 to 100 reintroduces the problem
with the next slightly longer header.

So ask the real question. `findVerdict()` is extracted from `parseVerdict()` and
returns the verdict a review STATES, or null — the distinction `parseVerdict`
structurally cannot express, since a stated COMMENT and no verdict at all come
back identical. It is the same distinction issue #20 needs; whoever picks that
up should reuse this rather than write a second one. `parseVerdict` keeps its
exact prior behaviour, now expressed as `findVerdict(output) ?? 'COMMENT'`.

The opencode lane then hard-fails a protocol-mode review with no verdict: a
reviewer that produced output but no verdict has not reviewed. General mode
(`--prompt`) is exempt — a question is not a review, and none was asked for.

Also from claude's review, all verified against the source first:

- a stale `CODEV_OPENCODE_BIN` said "install opencode" rather than naming the
  override that moved
- `listOpencodeModels` keeps only `provider/model`-shaped lines, so a future
  decorated listing degrades to "catalog unreadable" — handing authority back to
  the provider — instead of turning a valid id into a hard failure
- tests for the large-prompt temp-file branch, which had none
- `persistent-output.test.ts` and `lane-models.test.ts` lane loops extended

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Records the verdict requirement, the opencode lane's prerequisites and expected
timing, and — the part worth writing down — why `parseVerdict`'s length floor
was never the right guard. Someone reading only the fix would reasonably
conclude the number needs raising. It does not; it needs replacing, which
`findVerdict()` does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Caught by the opencode lane's confirming re-review: the lane table and flag docs
listed it, but consult.md's opening sentence and the skill's description and
one-liner still enumerated the old set. A reader who never scrolls past the
first paragraph would conclude the lane does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pseudoseed
pseudoseed merged commit 6adeb48 into main Aug 21, 2026
7 checks passed
pseudoseed added a commit that referenced this pull request Aug 21, 2026
pseudoseed added a commit that referenced this pull request Aug 21, 2026
The workspace config now names an 'opencode' consult lane, which the build on
this branch predates and rejects — so the branch's own codev CLI could not run,
including the forge subcommand this PR adds. Merging main resolves that and
keeps the PR current.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add opencode as a consult lane, which makes Grok available for review

1 participant