Skip to content

Feature request: support caching response_schema/structured-output config in CreateCachedContentConfig #2959

Description

@maheshetty

CreateCachedContentConfig currently accepts system_instruction, contents, tools, and tool_config, but not response_schema/response_mime_type. For prompts with large structured-output schemas, this means the schema is resent at full price on every call even when system_instruction is cached.

The only current workaround is converting the schema into a single forced-choice FunctionDeclaration/Tool and baking that + tool_config into the cache instead — which we validated works (schema tokens do show up under cached_content_token_count), but it comes with real downsides vs. native response_schema support:

  • Loses whatever constrained-decoding guarantee response_schema mode provides — function-calling isn't documented as giving the same schema-adherence guarantee.
  • Introduces MALFORMED_FUNCTION_CALL as a new failure mode (see Finish Message is Empty when Finish Reason is Malformed Function Call, even when it is populated in logs in AI Studio #1789) with no equivalent under response_schema mode.
  • Requires a full generate_content config swap (can't mix response_schema with tools/tool_config in the same cached request per current API validation), so cache-hit vs. cache-miss calls need materially different request shapes.

Feature request: allow response_schema/response_mime_type to be part of the cached content directly, so structured-output prompts get the same caching benefit as system-instruction-heavy ones without the function-calling workaround.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

priority: p3Desirable enhancement or fix. May not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions