Skip to content

fix: drop sampling knobs for Claude 4.7+ and clamp xhigh on 4.6 - #42

Merged
CMGS merged 3 commits into
mainfrom
batch0/sampling-and-effort-per-generation
Sep 2, 2026
Merged

fix: drop sampling knobs for Claude 4.7+ and clamp xhigh on 4.6#42
CMGS merged 3 commits into
mainfrom
batch0/sampling-and-effort-per-generation

Conversation

@CMGS

@CMGS CMGS commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Problem

claude_engine::build_body kept temperature / top_p (and top_k via raw extras) unless the gateway had itself mapped a reasoning request into thinking. That gate models Claude 4.6, where sampling parameters are accepted until thinking is on. Every later generation — Opus 4.7/4.8, Sonnet 5, Opus 5, Fable 5/5.1 — rejects the three parameters outright with a 400, and OpenAI-compatible clients send temperature by default, so a plain chat request to claude-opus-5 failed upstream.

map_thinking also passed the client's effort through verbatim, so reasoning_effort: xhigh (introduced with Opus 4.7) or an OpenRouter-style budget ≥ 20480 reached Claude 4.6 as output_config.effort: xhigh, which 4.6 does not accept. The live matrix already knew (its 4.6 tiers omit xhigh), the engine did not.

Change

  • Sampling knobs are dropped whenever the model's dialect is AdaptiveSummarized (4.7+ and the 5 family), and still only after a thinking mapping on Adaptive (4.6); the ≤4.5 budget dialect keeps them.
  • map_thinking clamps xhigh → high on the Adaptive dialect only.
  • docs/api.md states both rules; scripts/live-matrix gains a claude-opus-5 row with effort tiers including xhigh, a temperature-carrying chat call and a native adaptive+summarized stream, so the dialect every current Claude id resolves to has live coverage (none existed).

Evidence

  • Unit: sampling_knobs_follow_the_model_generation (4.5 and 4.6 keep the knobs, claude-opus-5 and claude-fable-5-1 drop all three) and xhigh_clamps_to_high_on_claude_4_6_only (effort and budget forms; 4.7+ keeps xhigh). Two integration tests that asserted the old passthrough on claude-fable-5 now assert the drop.
  • cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test (workspace): green. python3 -m py_compile on the live matrix: clean. The new live cases need vendor keys and were not run here.

Hot path: one anthropic_thinking_dialect string scan per request, which the reasoning branch already paid; no new allocation.

CMGS added 3 commits September 2, 2026 15:46
The Anthropic engine forwarded temperature, top_p and top_k whenever
the gateway had not itself mapped a thinking request, which was right
for Claude 4.6 and wrong for every later generation: Opus 4.7+, Sonnet
5, Opus 5 and the Fable family reject those parameters with a 400, and
OpenAI-compatible clients send a temperature by default. The knobs are
now dropped for the adaptive+summarized dialect regardless of thinking.

map_thinking passed the client's effort through verbatim, so xhigh (new
with Opus 4.7) reached Claude 4.6, which does not accept it; it clamps
to high there.

The live matrix gains a claude-opus-5 row with the effort tiers
including xhigh, a temperature-carrying chat call and a native adaptive
summarized stream, so the dialect every current Claude id resolves to
has live coverage.
@CMGS
CMGS merged commit e1c6265 into main Sep 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant