You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CreateCachedContentConfig currently accepts system_instruction, contents, tools, and tool_config, but not response_schema/response_mime_type. For prompts with large structured-output schemas, this means the schema is resent at full price on every call even when system_instruction is cached.
The only current workaround is converting the schema into a single forced-choice FunctionDeclaration/Tool and baking that + tool_config into the cache instead — which we validated works (schema tokens do show up under cached_content_token_count), but it comes with real downsides vs. native response_schema support:
Loses whatever constrained-decoding guarantee response_schema mode provides — function-calling isn't documented as giving the same schema-adherence guarantee.
Requires a full generate_content config swap (can't mix response_schema with tools/tool_config in the same cached request per current API validation), so cache-hit vs. cache-miss calls need materially different request shapes.
Feature request: allow response_schema/response_mime_type to be part of the cached content directly, so structured-output prompts get the same caching benefit as system-instruction-heavy ones without the function-calling workaround.
CreateCachedContentConfigcurrently acceptssystem_instruction,contents,tools, andtool_config, but notresponse_schema/response_mime_type. For prompts with large structured-output schemas, this means the schema is resent at full price on every call even whensystem_instructionis cached.The only current workaround is converting the schema into a single forced-choice
FunctionDeclaration/Tooland baking that +tool_configinto the cache instead — which we validated works (schema tokens do show up undercached_content_token_count), but it comes with real downsides vs. nativeresponse_schemasupport:response_schemamode provides — function-calling isn't documented as giving the same schema-adherence guarantee.MALFORMED_FUNCTION_CALLas a new failure mode (see Finish Message is Empty when Finish Reason is Malformed Function Call, even when it is populated in logs in AI Studio #1789) with no equivalent underresponse_schemamode.response_schemawithtools/tool_configin the same cached request per current API validation), so cache-hit vs. cache-miss calls need materially different request shapes.Feature request: allow
response_schema/response_mime_typeto be part of the cached content directly, so structured-output prompts get the same caching benefit as system-instruction-heavy ones without the function-calling workaround.