Skip to content

fix: drop a tool call that generation never finished - #742

Open
Javinator9889 wants to merge 1 commit into
ROCm:mainfrom
Javinator9889:fix/drop-truncated-tool-call
Open

Javinator9889 wants to merge 1 commit into
ROCm:mainfrom
Javinator9889:fix/drop-truncated-tool-call

Conversation

@Javinator9889

Copy link
Copy Markdown

Fixes #741.

When neither </tool_call> nor </function> is present, generation was cut off
inside the call — normally by the output limit while writing a large argument.
The parser took the remainder of the text as the block and emitted a
complete-looking tool call built from partial input:

} else {
    block_end = response_text.length();   // fabricates a complete call
}

The client cannot detect this, because the arguments JSON is well formed — it
is rebuilt from the parsed parameters, not copied from the raw text. A shell
heredoc cut mid-body becomes an unterminated cat <<EOF.

This drops the block instead, so no tool call is emitted. The response still
carries finish_reason: "length", which is the honest signal that the client
should retry with a larger budget. The </function> fallback is kept: a block
missing only the outer tag is complete.

Applied to the three models that share the pattern: qwen3_5vl,
qwen3_6_moe, qwen3_5_omni.

Verified on qwen3.5:9b, 300-line heredoc at max_tokens=400:

before (92f3f13) after
truncated tool_calls: true, 1580 chars of arguments, no closing EOF tool_calls: false, finish_reason: length
complete finish_reason: tool_calls, {"command":"echo hello"} unchanged

When neither </tool_call> nor </function> is present the call was cut off,
normally by the output limit while writing a large argument. Taking the
remainder of the text fabricates a complete-looking call from a partial one,
and the client cannot tell: the arguments JSON is well formed because it is
rebuilt from the parsed parameters. A shell heredoc cut mid-body becomes an
unterminated `cat <<EOF` that hangs whatever runs it.

Drop the block instead. The response still carries finish_reason "length",
which is the honest signal to retry with a larger budget. The </function>
fallback is kept: a block missing only the outer tag is complete.

Measured on qwen3.5:9b, a 300-line heredoc at max_tokens=400:
before, tool_calls=true with 1580 chars of arguments and no closing EOF;
after, tool_calls=false with finish_reason "length". A complete call is
unaffected: finish_reason "tool_calls", arguments {"command":"echo hello"}.

Refs ROCm#741
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Truncated tool call is emitted as a complete one (unterminated heredoc reaches the client)

1 participant