Skip to content

fix(openapi): decode error bodies with the declared charset - #7210

Open
1aifanatic wants to merge 1 commit into
google:mainfrom
1aifanatic:fix/rest-api-tool-error-body-charset
Open

1aifanatic wants to merge 1 commit into
google:mainfrom
1aifanatic:fix/rest-api-tool-error-body-charset

Conversation

@1aifanatic

Copy link
Copy Markdown

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem:

On a failed call, RestApiTool.call() built the model-facing error with response.content.decode("utf-8"). A non-UTF-8 error body, such as a Latin-1 page from a legacy server or a binary body, raised UnicodeDecodeError inside the except httpx.HTTPStatusError handler, where the sibling except ValueError can't catch it. The whole agent run then ended instead of the model getting the HTTP error.

Solution:

Use response.text, which is what the non-JSON success path a few lines below already does. It decodes with the response's declared charset and replaces undecodable bytes instead of raising.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.

  • All unit tests pass locally.

  • New test_call_http_failure_decodes_body_with_declared_charset: a real httpx.Response(404) with a charset=iso-8859-1 body is reported as Status Code: 404, Commande introuvable : échec. Fails on main with UnicodeDecodeError.

  • test_call_http_failure now sets the mock's .text instead of .content, because that's the attribute the tool reads.

$ pytest tests/unittests/tools/openapi_tool tests/unittests/tools/application_integration_tool     tests/unittests/tools/apihub_tool tests/unittests/tools/google_api_tool
531 passed

$ pytest tests/unittests -n 8   # Python 3.12, Windows 11
45 failed, 15143 passed, 102 skipped, 27 xfailed, 2 xpassed

Every failure also fails on an unmodified origin/main (3f4bb8f) on the same machine, where the same run gives 47 failed. Most are deterministic Windows-specific tests (cli/ deploy, conformance, scripts/check_new_py_files, path normalization, the unsafe local code executor, the import allowlist, yaml). The rest are skill-telemetry schema cases in telemetry/test_functional.py that fail intermittently on clean main here: rerun 3 times each, they failed 1, 1, 1 times on main and 1, 1, 3 times on this branch. I ran on Python 3.12 only, not the full tox matrix.

Manual End-to-End (E2E) Tests:

A real Runner + LlmAgent + OpenAPIToolset with a mocked model and an httpx.MockTransport backend that answers 404 with a Latin-1 body. The script is in #7206.

Before (main @ 3f4bb8f):

RUN FAILED: UnicodeDecodeError - 'utf-8' codec can't decode byte 0xe9 in position 23: invalid continuation byte

After:

function_response: {'error': "Tool get_order execution failed. ... Execution Error: Status Code: 404, Commande introuvable : échec"}

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

🤖 Generated with Claude Code

https://claude.ai/code/session_013vXxD1ga1hnCq2uFRwNks7

RestApiTool built the error message for a failed call with
response.content.decode("utf-8"). A non-UTF-8 error body (a Latin-1
page from a legacy server, a binary body) raised UnicodeDecodeError
inside the HTTPStatusError handler, where the sibling ValueError
handler cannot catch it, and ended the whole agent run instead of
reporting the HTTP error to the model.

Use response.text, as the non-JSON success path already does: it
honors the declared charset and replaces undecodable bytes.

Fixes google#7206

Claude-Session: https://claude.ai/code/session_013vXxD1ga1hnCq2uFRwNks7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RestApiTool crashes the agent run on a non-UTF-8 error response body (UnicodeDecodeError in the HTTPStatusError handler)

2 participants