test(opencode): use the config-defined model so bwrap tests stay hermetic - #129
Merged
Merged
Conversation
The opencode bwrap scenarios invoked '-m openai/o3-mini', which resolves against opencode's live-fetched model catalog (models.dev) — not the sha256-pinned binary. The catalog floated forward and dropped o3-mini, breaking main's integration (bwrap) job with three failure modes of the same dependency (pane.cast evidence from the CI artifacts): - catalog fetch fails -> 'UnknownError: Unexpected server error' right after the first-run DB migration (persists_streamed, server_500) - catalog fetch succeeds -> 'Model not found: openai/o3-mini. Did you mean: gpt-4.1-mini, gpt-4o-mini, gpt-5-mini?' (bash_tool_use) - catalog fetch falls back to the baked catalog -> test passes (streams_chunks, persists_user_prompt) All three failing tests saw zero requests reach the fake server because opencode crashed before its first model call. Fix: '-m openai/fake-model' — the model the scenario's own opencode.json defines under the provider's models map. Resolution never touches the live catalog, and the fake server echoes any model field back without validating it. Verified: cargo check -p agent-tui-integration --features bwrap --tests, cargo clippy --workspace --all-targets -- -D warnings, cargo fmt. The bwrap suite itself needs a container host — CI's integration (bwrap) job is the execution environment (same arrangement as #127). Note: crates/agent-tui/skill-data/ai-cli/SKILL.md still uses o3-mini in its (non-executed) real-world examples — stale for the same reason; left for a follow-up since the right replacement model for user-facing docs is a product call. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com>
pquerna
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Main's
integration (bwrap)job is red (since the #124 merge run — pre-existing, unrelated to #124): all threeopencode_fake_inferencefailures saw zero requests reach the fake inference server because opencode crashed before its first model call.Root cause, from the
pane.castCI artifacts: the scenarios invoke-m openai/o3-mini, which resolves against opencode's live-fetched model catalog (models.dev) — not the sha256-pinned v1.15.10 binary. The catalog floated forward and droppedo3-mini. One dependency, three failure modes depending on the fetch outcome:UnknownError: Unexpected server errorright after the first-run DB migration (opencode_persists_streamed_response_to_session_db,opencode_handles_server_500_without_hanging)Model not found: openai/o3-mini. Did you mean: gpt-4.1-mini, gpt-4o-mini, gpt-5-mini?(opencode_executes_bash_tool_use_and_persists_output)opencode_streams_chunks_incrementally,opencode_persists_user_prompt_in_session)Fix
-m openai/fake-model— the model each scenario's ownopencode.jsonalready defines under the provider'smodelsmap. Resolution then never touches the live catalog. The fake server echoes anymodelfield back without validating it, so request/response shapes are unchanged.Testing
cargo check -p agent-tui-integration --features bwrap --testscargo clippy --workspace --all-targets -- -D warnings,cargo fmt --checkintegration (bwrap)job is the execution environment (same arrangement as test(windows): ConPTY e2e coverage for the #124 parity fixes #127).Not in this PR
crates/agent-tui/skill-data/ai-cli/SKILL.mdstill useso3-miniin its real-world examples (the{test=…}blocks don't execute in CI today). Stale for the same root cause, but the right replacement model for user-facing docs is a product call — follow-up.🤖 Generated with Claude Code + Squire