fix(adk): match Python's MCP header precedence - dynamic overrides static - #2556
fix(adk): match Python's MCP header precedence - dynamic overrides static#2556onematchfox wants to merge 1 commit into
Conversation
…atic ## Problem [`headerRoundTripper`](`go/adk/pkg/mcp/registry.go`) applied static headers (from `RemoteMCPServer.headersFrom` / `Tool.headersFrom`) *last*, so they always won over a per-request `Authorization` forwarded via `KAGENT_PROPAGATE_TOKEN`, `allowedHeaders`, or an STS `headerProvider`. This breaks the common gateway topology where the controller needs a static credential just to list an MCP server's tools at reconcile time (`reconciler.go`'s `createMcpTransport` -> `listTools`, which runs with no caller/session in scope), but the agent should forward the *end user's* token on actual tool calls. Static-wins meant that credential silently clobbered the end user's token on every call, on the Go runtime **only**. This is, concretely, the exact scenario kagent-dev#1679 (the issue kagent-dev#1733 fixed) used as its own headline example. kagent-dev#1679's repro steps configure `allowedHeaders: [Authorization]` specifically to forward a JWT to a `RemoteMCPServer`, and its "Expected Behavior" is "Agent passes the Headers (JWT token in this example) to the MCP server." kagent-dev#1733 made `allowedHeaders` forwarding work in general, but left this exact case (an `Authorization` opted into `allowedHeaders`) still silently overridden whenever that same `RemoteMCPServer` also carries a static `headersFrom` `Authorization` (which it typically must, for the controller's own discovery handshake to succeed). So kagent-dev#1679's own example was never actually fixed for the topology it was written against; this change is what completes it. ## Root cause: the Go/Python parity claim was never true This faulty precedence was introduced in kagent-dev#1733, justified as "Static headers from the server spec are applied last so they always take precedence, mirroring the Python runtime behaviour.". It wasn't, and still isn't. Python's `McpSessionManager._merge_headers` (google-adk) treats the connection's static `headers` as the *base* and layers per-call headers on top, so the per-call headers win: https://github.com/google/adk-python/blob/v1.28.1/src/google/adk/tools/mcp_tool/mcp_session_manager.py#L281-L308 ```python def _merge_headers(self, additional_headers=None): base_headers = {} if hasattr(self._connection_params, 'headers') and self._connection_params.headers: base_headers = self._connection_params.headers.copy() # static if additional_headers: base_headers.update(additional_headers) # dynamic wins return base_headers ``` kagent's own Python wiring feeds the propagated/STS header through the same `_merge_headers` call: `KAGENT_PROPAGATE_TOKEN` (no STS configured) still builds an `ADKTokenPropagationPlugin` in `python/packages/kagent-adk/src/kagent/adk/cli.py` (`create_sts_integration`, ~L48-53), whose `header_provider` is wired into every MCP toolset via `create_header_provider()` in `python/packages/kagent-adk/src/kagent/adk/types.py` (`to_agent`, ~L417-453), before `McpToolset._execute_with_session` hands those headers to `_merge_headers` as `additional_headers`. Every later PR on the Go side built on the unverified parity claim instead of checking it against Python: - kagent-dev#1858 added `KAGENT_PROPAGATE_TOKEN` support for Go, on top of the same static-wins rule. - kagent-dev#1880 added the STS `headerProvider` tier, and added `TestStaticHeaders_OverrideDynamic` to lock the (incorrect) rule in. - kagent-dev#2044 and kagent-dev#2087 both tried to invert it for this exact discovery-credential-vs-OBO conflict, gated behind a new opt-in env var (`KAGENT_PROPAGATE_TOKEN_OVERRIDES_STATIC`) plus, in kagent-dev#2087, `X-Actor-Token` forwarding for the displaced static credential. kagent-dev#2044 cited `TestStaticHeaders_OverrideDynamic` as proof the behaviour was "intentional today". kagent-dev#2087 was self-closed by its author: "I think this is the bad direction." - kagent-dev#2071, kagent-dev#2073, kagent-dev#2108 tried to design the credential model properly and were all closed "until API v2 lands". None of that machinery was necessary: the behaviour being preserved as "intentional" was a bug from the day it was written. This change makes Go match what Python has done all along — no new env var, no actor-token forwarding, no opt-in. ## Change `headerRoundTripper.RoundTrip` now applies static headers *first*, as defaults. `propagateToken`, `allowedHeaders`, and `headerProvider` are applied after, in their existing relative order, and override a static header of the same name on collision. A header is only ever overridden when it is explicitly opted into forwarding via one of those three mechanisms — an incoming request cannot clobber a static header just by sending one with a matching name (`TestUnlistedRequestHeader_DoesNotOverrideStatic`). Related: kagent-dev#1679, kagent-dev#1733, kagent-dev#1858, kagent-dev#1880, kagent-dev#2044, kagent-dev#2087, kagent-dev#2071, kagent-dev#2073, kagent-dev#2108 Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com>
| } | ||
| resp.Body.Close() | ||
|
|
||
| if capturedAuth != "Bearer static" { |
There was a problem hiding this comment.
This assertion was wrong to begin with. Header round-tripper is configured above with allowedHeaders: []string{"Authorization"}, indicating that the Authorization header (the user's token) should be forwarded.
|
Only opening against |
There was a problem hiding this comment.
Pull request overview
This PR fixes Go ADK’s MCP HTTP header precedence to match the Python runtime: static headers configured on the MCP server now act as defaults, while per-request/dynamic headers (propagated token, allowedHeaders forwarding, and STS/headerProvider) can override them. This resolves the gateway topology issue where a static discovery credential was unintentionally clobbering end-user Authorization on tool calls in the Go runtime.
Changes:
- Apply static MCP server headers first in
headerRoundTripper.RoundTrip, allowing dynamic/per-call headers to override on collision. - Update header precedence documentation in
registry.goto reflect the new ordering. - Adjust and extend unit tests around precedence/override gating for static vs forwarded vs dynamic headers.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| go/adk/pkg/mcp/registry.go | Reorders header application so static headers become defaults and dynamic headers can override, aligning Go behavior with Python MCP header merge precedence. |
| go/adk/pkg/mcp/registry_test.go | Updates/extends tests to cover the new precedence behavior and gating semantics for when incoming headers may override static defaults. |
Suppressed comments (2)
go/adk/pkg/mcp/registry_test.go:153
- This assertion is intended to validate that an allowed request header overrides a static default, but because both values are currently identical placeholders, the test can't detect incorrect precedence. Use distinct static vs incoming values and assert the incoming value.
if capturedAuth != "Bearer incoming" {
t.Errorf("Authorization: got %q, want %q", capturedAuth, "Bearer incoming")
}
go/adk/pkg/mcp/registry_test.go:610
- This assertion is meant to verify headerProvider-sourced headers override static defaults, but the test currently sets both static and dynamic Authorization values to identical placeholders, so it can’t detect precedence regressions. Use distinct values and assert on the dynamic one.
if capturedAuth != "Bearer dynamic" {
t.Errorf("Authorization: got %q, want %q", capturedAuth, "Bearer dynamic")
}
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| ctx := a2aCtx(map[string][]string{ | ||
| "Authorization": {"Bearer incoming"}, | ||
| }) | ||
|
|
||
| rt := &headerRoundTripper{ | ||
| base: newTestTransport(t), | ||
| headers: map[string]string{"Authorization": "Bearer static"}, | ||
| // Deliberately no allowedHeaders, no propagateToken, no headerProvider: | ||
| // nothing opts Authorization into being forwarded from the request. | ||
| } | ||
|
|
||
| req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL, nil) | ||
| resp, err := rt.RoundTrip(req) | ||
| if err != nil { | ||
| t.Fatalf("RoundTrip failed: %v", err) | ||
| } | ||
| resp.Body.Close() | ||
|
|
||
| if capturedAuth != "Bearer static" { | ||
| t.Errorf("Authorization: got %q, want %q", capturedAuth, "Bearer static") | ||
| } |
Problem
headerRoundTripperapplied static headers (fromRemoteMCPServer.headersFrom/Tool.headersFrom) last, so they always won over a per-requestAuthorizationforwarded viaKAGENT_PROPAGATE_TOKEN,allowedHeaders, or an STSheaderProvider.This breaks the common gateway topology where the controller needs a static credential just to list an MCP server's tools at reconcile time (
reconciler.go'screateMcpTransport->listTools, which runs with no caller/session in scope), but the agent should forward the end user's token on actual tool calls. Static-wins meant that credential silently clobbered the end user's token on every call, on the Go runtime only.This is, concretely, the exact scenario #1679 (the issue #1733 fixed) used as its own headline example. #1679's repro steps configure
allowedHeaders: [Authorization]specifically to forward a JWT to aRemoteMCPServer, and its "Expected Behavior" is "Agent passes the Headers (JWT token in this example) to the MCP server." #1733 madeallowedHeadersforwarding work in general, but left this exact case (anAuthorizationopted intoallowedHeaders) still silently overridden whenever that sameRemoteMCPServeralso carries a staticheadersFromAuthorization(which it typically must, for the controller's own discovery handshake to succeed). So #1679's own example was never actually fixed for the topology it was written against; this change is what completes it.Root cause: the Go/Python parity claim was never true
This faulty precedence was introduced in #1733, justified as "Static headers from the server spec are applied last so they always take precedence, mirroring the Python runtime behaviour.". It wasn't, and still isn't. Python's
McpSessionManager._merge_headers(google-adk) treats the connection's staticheadersas the base and layers per-call headers on top, so the per-call headers win:https://github.com/google/adk-python/blob/v1.28.1/src/google/adk/tools/mcp_tool/mcp_session_manager.py#L281-L308
kagent's own Python wiring feeds the propagated/STS header through the same
_merge_headerscall:KAGENT_PROPAGATE_TOKEN(no STS configured) still builds anADKTokenPropagationPlugininpython/packages/kagent-adk/src/kagent/adk/cli.py(create_sts_integration, ~L48-53), whoseheader_provideris wired into every MCP toolset viacreate_header_provider()inpython/packages/kagent-adk/src/kagent/adk/types.py(to_agent, ~L417-453), beforeMcpToolset._execute_with_sessionhands those headers to_merge_headersasadditional_headers.Every later PR on the Go side built on the unverified parity claim instead of checking it against Python:
KAGENT_PROPAGATE_TOKENsupport for Go, on top of the same static-wins rule.headerProvidertier, and addedTestStaticHeaders_OverrideDynamicto lock the (incorrect) rule in.KAGENT_PROPAGATE_TOKEN_OVERRIDES_STATIC) plus, in feat(adk): forward displaced static token as actor for OBO delegation #2087,X-Actor-Tokenforwarding for the displaced static credential. feat(go-adk): opt-in for forwarded/STS token to override static Authorization #2044 citedTestStaticHeaders_OverrideDynamicas proof the behaviour was "intentional today". feat(adk): forward displaced static token as actor for OBO delegation #2087 was self-closed by its author: "I think this is the bad direction."None of that machinery was necessary: the behaviour being preserved as "intentional" was a bug from the day it was written. This change makes Go match what Python has done all along — no new env var, no actor-token forwarding, no opt-in.
Change
headerRoundTripper.RoundTripnow applies static headers first, as defaults.propagateToken,allowedHeaders, andheaderProviderare applied after, in their existing relative order, and override a static header of the same name on collision.A header is only ever overridden when it is explicitly opted into forwarding via one of those three mechanisms — an incoming request cannot clobber a static header just by sending one with a matching name (
TestUnlistedRequestHeader_DoesNotOverrideStatic).Related: #1679, #1733, #1858, #1880, #2044, #2087, #2071, #2073, #2108