google realtime: MCP tool schema with `readOnly`/`x-google-enum-descriptions` kills the whole session (config built outside try)

[caiolea0]

Summary

A single MCP tool whose input schema uses standard JSON Schema annotation keywords (readOnly, x-google-enum-descriptions, …) makes the Gemini Realtime plugin fail to build its connect config, which kills the realtime session permanently — the agent joins the room, RoomIO links to the participant, and then neither hears nor speaks. No retry, no fallback, nothing visible to the end user.

Versions

• livekit-agents 1.8.2 (= latest on PyPI)
• livekit-plugins-google 1.8.2 (= latest on PyPI)
• google-genai 2.18.1, pydantic 2.13.4
• model: gemini-3.1-flash-live-preview (Gemini Realtime)

Also reproducible/valid by reading main today: _build_connect_config() is still called outside the try (line 971 vs 973) and _GeminiJsonSchema._simplify() still has no unknown-key handling (line 151+).

What happens

ERROR livekit.plugins.google Error in _main_task
pydantic_core._pydantic_core.ValidationError: 1 validation error for FunctionDeclaration
parameters.properties.attachments.items.properties.id.readOnly
  Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool]

The tools came from Google's own hosted Gmail MCP server (https://gmailmcp.googleapis.com/mcp/v1), attached through mcp_servers= on AgentSession. 4 of the 5 tools I allow-listed failed the same way:

tool	offending keyword
create_draft	readOnly: true on $defs.Attachment.properties.id
get_thread, get_message	x-google-enum-descriptions on messageFormat
search_threads	x-google-enum-descriptions on view

Both keywords are legitimate JSON Schema (readOnly is spec; x-google-enum-descriptions is a Google extension), so any MCP server can produce them — this is not Gmail-specific.

Root cause (two defects)

1. _GeminiJsonSchema.simplify() doesn't drop unknown keywords. realtime_api._build_connect_config() calls create_tools_config(..., use_parameters_json_schema=False), so raw tools (which is what MCP tools are — RawFunctionTool) go through simplify(). It pops title/default/additionalProperties/$schema/const/discriminator/examples and resolves $ref, but anything else survives straight into types.FunctionDeclaration.model_validate(), whose Schema model is extra="forbid".

2. The config is built outside the try in _main_task. The exception escapes the while loop before any connection attempt, so there is no retry and no session: the worker sits in the room, publishes mic, and stays silent forever. A tool-schema problem should degrade the tool, not silence the agent.

Minimal repro (no LiveKit needed)

from google.genai import types
from livekit.plugins.google.utils import _GeminiJsonSchema

# exactly what the Gmail MCP returns for search_threads.view
schema = {
    "type": "object",
    "properties": {
        "view": {
            "type": "string",
            "enum": ["THREAD_VIEW_UNSPECIFIED", "THREAD_VIEW_MINIMAL"],
            "x-google-enum-descriptions": ["a", "b"],
        }
    },
}

params = _GeminiJsonSchema(schema).simplify()  # keeps x-google-enum-descriptions
types.FunctionDeclaration.model_validate(
    {"name": "t", "description": "", "parameters": params}
)
# pydantic_core._pydantic_core.ValidationError: Extra inputs are not permitted ...

For readOnly the same thing happens one level deeper, inside a $defs entry reached through $ref:

schema = {
    "type": "object",
    "properties": {
        "attachments": {
            "type": "array",
            "items": {"$ref": "#/$defs/Attachment"},
        }
    },
    "$defs": {
        "Attachment": {
            "type": "object",
            "properties": {"id": {"type": "string", "readOnly": True}},
        }
    },
}

Suggested fix

• In _GeminiJsonSchema._simplify, drop keywords the Gemini Schema doesn't know (its field set is a ready-made allowlist: types.Schema.model_fields) — these are annotations and carry no meaning for the model.
• In realtime_api, build the connect config inside the try (or catch per-session and log), so a bad tool can't take the audio down with it. Dropping the offending declaration with a logger.warning would be even better than dropping the session.

I can open a PR for both if you'd like, but I wanted to file the report first since it's a session-level failure that's invisible from the client side.

Context

Friday (a Brazilian Portuguese voice assistant built on LiveKit Agents) went completely deaf right after the owner enabled the Gmail MCP in its capability list — 6 start attempts, all dead, with the only clue buried in the worker log. Workaround on my side was sanitizing MCP tool schemas before the plugin sees them; the underlying defects are the two above.

[rokasremeika]

The realtime provider treats the setup payload as malformed when it contains JSON Schema keys it does not explicitly support, and closes the websocket almost immediately with close code 1007. Because it closes for invalid frame payload data rather than returning a field-level error, there is no validation message or useful error text. Application code never sees the frame, so the session produces no audio at all while the room still shows a healthy participant. To fix this, patch the plugin's setup-config construction to emit the exact schema the provider accepts by stripping out unsupported keys. Apply this as an idempotent runtime patch guarded by a sentinel attribute on the patched class so a re-import or double application is a no-op. To verify the fix, patch the provider client's session-connect entry point to log the outbound setup payload before it goes on the wire. A 1007 tells you nothing on its own, but inspecting the logged bytes will confirm the rejected keys are gone and you can measure success by watching the websocket stay open.