Skip to content

fix(providers): send maxOutputTokens under the name the SDK reads - #1335

Merged
will-lamerton merged 1 commit into
mainfrom
fix/provider-max-output-tokens
Sep 16, 2026
Merged

will-lamerton merged 1 commit into
mainfrom
fix/provider-max-output-tokens

Conversation

@will-lamerton

Copy link
Copy Markdown
Member

The bug

The AI SDK renamed this setting to maxOutputTokens in v5. We're on ai@6.0.193 and still sending maxTokens (chat-handler.ts). The v6 typings contain 8 references to maxOutputTokens and zero maxTokens: declarations, so the value was being dropped on the floor.

It typechecked because the call site spreads it:

...(modeOverrides?.modelParameters && {
  maxTokens: modeOverrides.modelParameters.maxTokens,  // silently ignored
}),

Object spreads bypass excess-property checking. No compile error, no runtime error, no log - the ceiling just quietly became whatever the provider inferred from the model id.

/tune's max-tokens control has therefore been inert for every provider and every user since the v6 upgrade.

Why it's worse than it sounds

@ai-sdk/anthropic derives the ceiling from the model id, with this fallback (dist/index.mjs:5222):

} else {
  return { maxOutputTokens: 4096, ..., isKnownModel: false };
}

Point sdkProvider: "anthropic" at an Anthropic-compatible endpoint serving a non-Claude model and every reply is capped at 4096 tokens, truncated mid-sentence with no error.

That is what has been breaking nc-review on Nano-Collective/prompt-scrubber: reviews died mid-analysis without ever writing their verdict. nanocoder#1332 made the truncation visible and recoverable; this removes the cause.

Changes

  • Send the setting as maxOutputTokens, restoring /tune.
  • Add maxOutputTokens to provider entries in agents.config.json. Headless runs build modeOverrides without modelParameters, so /tune is not a route CI can use - the provider entry is. /tune still wins where set.
  • The user-facing name stays maxTokens in ModelParameters; it's persisted in tune preferences, so renaming would silently discard existing user config. The SDK name is applied at the boundary.

Behaviour change worth flagging

Fixing this means tune values start applying. The Nano (low-end hardware) preset sets maxTokens: 2048, which until now did nothing and will now actually cap output. That is the documented intent of the preset, but anyone on it will notice shorter replies.

Tests

Three in chat-handler.spec.ts, asserting the request body over an intercepted fetch rather than trusting types - typechecking demonstrably cannot catch this class of bug:

  • an unrecognised model id yields max_tokens: 4096 (pins the default that makes this necessary)
  • maxOutputTokens: 32000 reaches the wire as max_tokens: 32000
  • maxTokens: 32000 does not - a guard that fails loudly if the SDK ever renames back

Plus maxOutputTokens plumbing asserted in client-factory.spec.ts.

chat-handler.spec.ts (29), client-factory.spec.ts (43), config-schema.spec.ts (47) pass; tsc --noEmit and biome ci clean. I did not run the full ava suite locally.

🤖 Generated with Claude Code

The AI SDK renamed this setting to maxOutputTokens in v5. Nanocoder was
still sending the v4 name maxTokens, and because object spreads bypass
excess-property checking it was dropped silently rather than failing to
compile. Every request fell back to whatever ceiling the provider
inferred from the model id, and /tune's max-tokens control has been inert
since the v6 upgrade.

That is worst on sdkProvider: 'anthropic', where @ai-sdk/anthropic reads
the ceiling off the model id and falls back to 4096 for anything it does
not recognise as a Claude model. An Anthropic-compatible endpoint serving
some other model had every reply truncated at 4096 with no error.

Also add maxOutputTokens to provider entries in agents.config.json.
Headless runs build modeOverrides without modelParameters, so /tune is
not a route CI can use; the provider entry is. A /tune value still wins
where one is set, and the user-facing name stays maxTokens because it is
persisted in tune preferences.

Tests assert what reaches the wire rather than what typechecks, since
typechecking demonstrably could not catch this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added the area:docs Documentation label Sep 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

nc-review: nothing to raise

@will-lamerton — nothing to raise from the automated review.

Renames the SDK-bound ceiling setting from maxTokens (v4, silently dropped by ai@6) to maxOutputTokens, threads maxOutputTokens through loadProviderConfigs, the schema and the type, and resolves /tune's maxTokens to the SDK name at the boundary so persisted preferences are not invalidated. Three new wire-level tests pin the SDK contract (default 4096 for unrecognised Anthropic model ids, maxOutputTokens reaching the wire as max_tokens, and the v4 spelling being ignored) and one client-factory test asserts plumbing. No findings.


🔴 blocking · 🟠 a reviewer would ask for a change · ⚪ optional

Automated code review — correctness, security, design, tests, plus duplicates and scope. A human still decides; this is not a substitute for review and is not exhaustive. The required status checks separately cover lint, formatting, types, unused dependencies, the test suite and the build. This bot never merges. Maintainers can rerun with /re-review.

@github-actions github-actions Bot added the agent:clean nc-review had nothing to raise label Sep 16, 2026
@will-lamerton
will-lamerton merged commit c87bee1 into main Sep 16, 2026
17 checks passed
will-lamerton added a commit to Nano-Collective/.github that referenced this pull request Sep 16, 2026
@ai-sdk/anthropic reads the ceiling off the model id and falls back to
4096 for anything that is not a known Claude model. `minimax-m3` is not
one, so every review turn was capped at 4096 tokens and stopped
mid-sentence, having never called write_file. That is the whole of
"nc-review could not produce a verdict this run": three runs on
prompt-scrubber died that way, twice on the same PR.

Nothing in the logs said so. A truncated turn arrives as content with no
tool calls, which is what a finished turn looks like.

Default 32000, deliberately conservative. MiniMax documents 131072 as the
recommended output limit and OpenRouter reports 262144 for the same
model; the numbers disagree across providers and an oversized value is
rejected outright by some endpoints rather than clamped. A verdict is a
few thousand tokens, so there is nothing to buy by going near either.

Guarded with jq rather than trusted: a missing ceiling does not fail the
run, it just silently truncates, and that is expensive to diagnose from a
transcript that simply stops.

Requires Nano-Collective/nanocoder#1335 for the setting to be read. Until
that lands the key is ignored, so this is safe to land first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent:clean nc-review had nothing to raise area:docs Documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant