Gemini 3.8 Live is async-by-default: plugin ends up in a hybrid blocking/non-blocking state, behavior/scheduling cannot be set per tool, and tool results wait for playout

[IngLP]

Summary

gemini-3.8-live (GA, 2026-09-15) makes asynchronous function calling the default: "Async execution (behavior: NON_BLOCKING) is now the default function calling mode. You can still use synchronous blocking mode for backwards compatibility by setting behavior: BLOCKING on your tool declarations" (model page, "Migrating from Gemini 3.1 Flash Live"). gemini-3.8-live-extended-thinking accepts only NON_BLOCKING. Three things in livekit-agents / livekit-plugins-google (main fc22671, identical to 1.8.2 for the files below) get in the way of using that properly. Measured on 1.8.1 with the production-size system instruction of a phone agent, 15 live trials per arm.

1. With the new default, the plugin ends up in a hybrid state it does not know about

tool_behavior is not set → to_fnc_ctx omits behavior from every FunctionDeclaration (livekit-agents/livekit/agents/llm/_provider_format/google.py:152, :167: if tool_behavior is not None: schema["behavior"] = ...). On 3.8 the server then treats every tool as NON_BLOCKING, but supports_silent_scheduling is derived from what the client set (livekit-plugins-google/livekit/plugins/google/realtime/realtime_api.py:708-711), so it stays False: a tool result with reply_required=False (typical for a fire-and-forget update, or for a tool that completed inside an interrupted turn, voice/generation.py marks those) is sent without scheduling: SILENT, the model answers anyway, and the warning at realtime_api.py:718-723 ("a tool result wants no reply … declare the tools NON_BLOCKING on the Gemini API to keep it silent") fires on every call: 15/15 and 15/15 in two arms with the default, 10/10 with tool_behavior=BLOCKING set explicitly, 0/15 with tool_behavior=NON_BLOCKING set explicitly. Suggestion: on models where the documented default is NON_BLOCKING, either send behavior explicitly (whatever the user chose, defaulting to the model's default) or derive supports_silent_scheduling from the effective behavior, not from the option.

2. behavior and scheduling are per session, they need to be per tool

RealtimeModel(tool_behavior=…, tool_response_scheduling=…) writes the same behavior on every FunctionDeclaration (_provider_format/google.py:152-168) and the same scheduling on every FunctionResponse (livekit-plugins-google/livekit/plugins/google/utils.py:62-110). A real agent needs both kinds at once: a CRM update or a "mark call successful" is NON_BLOCKING + SILENT (nothing to say about it), a calendar lookup whose result is the next sentence is BLOCKING, a hangup must be BLOCKING because with NON_BLOCKING the model may keep talking after calling it and the speech is cut by the hangup. FunctionToolInfo / RawFunctionToolInfo (llm/tool_context.py:184-214) carry no such field. #2367 asked for exactly @function_tool(behavior="NON_BLOCKING") and was closed as completed by #3482, which shipped only the session-wide option. A plugin-level map like the one #7197 just gave the Phonic plugin (configs_for_tools, tool name → config) would be enough and would not touch the core.

3. The FunctionResponse is held until all queued speech has been played

voice/agent_activity.py:4667-4676 (while self._current_speech or self._speech_q, comment "most realtime models don't support generating multiple responses at the same time") sends the tool result only after playout. For an async tool this defeats the point: scheduling: INTERRUPT can never interrupt anything, WHEN_IDLE arrives late, and the model's own decision to keep talking while the tool runs is followed by a result that lands after the talking is over. The AsyncToolset path has the same wait (voice/tool_executor.py:541-550, wait_for_idle() before generate_reply). #6741 (open) touches the adjacent problem of keeping the generation open after a NON_BLOCKING call; the send-after-playout is untouched. On 3.8, where the model is designed around this, the result should be sent as soon as the tool returns, with the scheduling telling the model what to do with it.

Why it matters, numbers

Same agent, same script that asks the model to persist a value mid-call, gemini-3.8-live, 15 trials per arm: with NON_BLOCKING + WHEN_IDLE the model performs the mid-call save in 7/15 trials with a median of 5.1 s from the end of the caller's sentence to the toolCall and 0 unanswered caller turns; with BLOCKING set explicitly 3/10 saves and 16.8 s; on gemini-3.1-flash-live-preview (sync only) 4/10 and 19.2 s. Async is the better mode on 3.8 — but today it can only be turned on for all tools at once, including hangup_call and the end-of-call finalization tool, and its result still waits for playout.

Environment

livekit-agents 1.8.1, livekit-plugins-google 1.8.1, google-genai 2.22.0, Python 3.12, models gemini-3.8-live, gemini-3.8-live-extended-thinking, gemini-3.1-flash-live-preview (API key). Line numbers above are from main at fc22671. Related: google-gemini/gemini-live-api-examples#46 (the model-side repetition before a function call, measured with the same harness).

[IngLP]

A fourth item, measured: the Gemini realtime path declares free-form object parameters with no shape, and it costs repeated speech. In livekit-plugins-google, _GeminiJsonSchema._simplify does schema.pop("additionalProperties", None) (utils.py:154) and simplify() returns None for an OBJECT with no properties (utils.py:146-149); the llm.FunctionTool branch of to_fnc_ctx ignores the use_parameters_json_schema flag entirely and always emits the simplified parameters (livekit-agents/livekit/agents/llm/_provider_format/google.py:156-169 — only the RawFunctionTool branch honours it); and the realtime session passes use_parameters_json_schema=False anyway (google/realtime/realtime_api.py:1255). Net effect on 1.8.2: a dict[str, Any] tool argument reaches Gemini Live as a bare {"type": "OBJECT"}.

Contrary to the comment at _provider_format/google.py:144-145, Gemini Live does accept parameters_json_schema — we probed the setup handshake directly on gemini-3.1-flash-live-preview and gemini-3.8-live, both accept it, and additionalProperties inside parameters is what gets rejected with a 1007. With the faithful schema sent that way, in a paired A/B of 40 live calls that differ in nothing else (production-size system instruction, thinkingLevel=LOW, same caller audio, same instant): verbatim sentence repetition 0/20 vs 10/20, silent gaps inside a generation 0.42 s worst vs 3.59 s, audio before the toolCall 4.6 s median vs 16.6 s, and the final tool call arriving 20/20 vs 17/20. The stripped shape also loses the argument itself: a separate bare-SDK probe on gemini-3.8-live with a production-size instruction had the model answer {} for such a parameter in 7 runs out of 10, against 10/10 filled with the faithful schema. Suggestion: honour use_parameters_json_schema in the FunctionTool branch and default the realtime path to True, plus a test that pins the emitted setup payload — none of the existing tests would have caught either the stripping or the 1007, since they all pin _GeminiJsonSchema in isolation. Details and raw numbers in google-gemini/gemini-live-api-examples#46.