Skip to content

fix(adk): match Python's MCP header precedence - dynamic overrides static - #2556

Open
onematchfox wants to merge 1 commit into
kagent-dev:release/v0.10.xfrom
onematchfox:fix-golang-header-precedence
Open

fix(adk): match Python's MCP header precedence - dynamic overrides static#2556
onematchfox wants to merge 1 commit into
kagent-dev:release/v0.10.xfrom
onematchfox:fix-golang-header-precedence

Conversation

@onematchfox

Copy link
Copy Markdown
Contributor

Problem

headerRoundTripper applied static headers (from RemoteMCPServer.headersFrom / Tool.headersFrom) last, so they always won over a per-request Authorization forwarded via KAGENT_PROPAGATE_TOKEN, allowedHeaders, or an STS headerProvider.

This breaks the common gateway topology where the controller needs a static credential just to list an MCP server's tools at reconcile time (reconciler.go's createMcpTransport -> listTools, which runs with no caller/session in scope), but the agent should forward the end user's token on actual tool calls. Static-wins meant that credential silently clobbered the end user's token on every call, on the Go runtime only.

This is, concretely, the exact scenario #1679 (the issue #1733 fixed) used as its own headline example. #1679's repro steps configure allowedHeaders: [Authorization] specifically to forward a JWT to a RemoteMCPServer, and its "Expected Behavior" is "Agent passes the Headers (JWT token in this example) to the MCP server." #1733 made allowedHeaders forwarding work in general, but left this exact case (an Authorization opted into allowedHeaders) still silently overridden whenever that same RemoteMCPServer also carries a static headersFrom Authorization (which it typically must, for the controller's own discovery handshake to succeed). So #1679's own example was never actually fixed for the topology it was written against; this change is what completes it.

Root cause: the Go/Python parity claim was never true

This faulty precedence was introduced in #1733, justified as "Static headers from the server spec are applied last so they always take precedence, mirroring the Python runtime behaviour.". It wasn't, and still isn't. Python's McpSessionManager._merge_headers (google-adk) treats the connection's static headers as the base and layers per-call headers on top, so the per-call headers win:

https://github.com/google/adk-python/blob/v1.28.1/src/google/adk/tools/mcp_tool/mcp_session_manager.py#L281-L308

def _merge_headers(self, additional_headers=None):
    base_headers = {}
    if hasattr(self._connection_params, 'headers') and self._connection_params.headers:
      base_headers = self._connection_params.headers.copy()   # static
    if additional_headers:
      base_headers.update(additional_headers)                  # dynamic wins
    return base_headers

kagent's own Python wiring feeds the propagated/STS header through the same _merge_headers call: KAGENT_PROPAGATE_TOKEN (no STS configured) still builds an ADKTokenPropagationPlugin in
python/packages/kagent-adk/src/kagent/adk/cli.py (create_sts_integration, ~L48-53), whose header_provider is wired into every MCP toolset via create_header_provider() in
python/packages/kagent-adk/src/kagent/adk/types.py (to_agent, ~L417-453), before McpToolset._execute_with_session hands those headers to _merge_headers as additional_headers.

Every later PR on the Go side built on the unverified parity claim instead of checking it against Python:

None of that machinery was necessary: the behaviour being preserved as "intentional" was a bug from the day it was written. This change makes Go match what Python has done all along — no new env var, no actor-token forwarding, no opt-in.

Change

headerRoundTripper.RoundTrip now applies static headers first, as defaults. propagateToken, allowedHeaders, and headerProvider are applied after, in their existing relative order, and override a static header of the same name on collision.

A header is only ever overridden when it is explicitly opted into forwarding via one of those three mechanisms — an incoming request cannot clobber a static header just by sending one with a matching name (TestUnlistedRequestHeader_DoesNotOverrideStatic).

Related: #1679, #1733, #1858, #1880, #2044, #2087, #2071, #2073, #2108

…atic

## Problem

[`headerRoundTripper`](`go/adk/pkg/mcp/registry.go`) applied static headers
(from `RemoteMCPServer.headersFrom` / `Tool.headersFrom`) *last*, so they
always won over a per-request `Authorization` forwarded via
`KAGENT_PROPAGATE_TOKEN`, `allowedHeaders`, or an STS `headerProvider`.

This breaks the common gateway topology where the controller needs a static
credential just to list an MCP server's tools at reconcile time
(`reconciler.go`'s `createMcpTransport` -> `listTools`, which runs with no
caller/session in scope), but the agent should forward the *end user's* token
on actual tool calls. Static-wins meant that credential silently clobbered
the end user's token on every call, on the Go runtime **only**.

This is, concretely, the exact scenario kagent-dev#1679 (the issue kagent-dev#1733 fixed) used as
its own headline example. kagent-dev#1679's repro steps configure
`allowedHeaders: [Authorization]` specifically to forward a JWT to a
`RemoteMCPServer`, and its "Expected Behavior" is "Agent passes the Headers
(JWT token in this example) to the MCP server." kagent-dev#1733 made `allowedHeaders`
forwarding work in general, but left this exact case (an `Authorization`
opted into `allowedHeaders`) still silently overridden whenever that same
`RemoteMCPServer` also carries a static `headersFrom` `Authorization` (which
it typically must, for the controller's own discovery handshake to succeed).
So kagent-dev#1679's own example was never actually fixed for the topology it was
written against; this change is what completes it.

## Root cause: the Go/Python parity claim was never true

This faulty precedence was introduced in kagent-dev#1733, justified as "Static headers
from the server spec are applied last so they always take precedence, mirroring
the Python runtime behaviour.". It wasn't, and still isn't. Python's
`McpSessionManager._merge_headers` (google-adk) treats the connection's
static `headers` as the *base* and layers per-call headers on top, so the
per-call headers win:

https://github.com/google/adk-python/blob/v1.28.1/src/google/adk/tools/mcp_tool/mcp_session_manager.py#L281-L308

```python
def _merge_headers(self, additional_headers=None):
    base_headers = {}
    if hasattr(self._connection_params, 'headers') and self._connection_params.headers:
      base_headers = self._connection_params.headers.copy()   # static
    if additional_headers:
      base_headers.update(additional_headers)                  # dynamic wins
    return base_headers
```

kagent's own Python wiring feeds the propagated/STS header through the same
`_merge_headers` call: `KAGENT_PROPAGATE_TOKEN` (no STS configured) still
builds an `ADKTokenPropagationPlugin` in
`python/packages/kagent-adk/src/kagent/adk/cli.py` (`create_sts_integration`,
~L48-53), whose `header_provider` is wired into every MCP toolset via
`create_header_provider()` in
`python/packages/kagent-adk/src/kagent/adk/types.py` (`to_agent`, ~L417-453),
before `McpToolset._execute_with_session` hands those headers to
`_merge_headers` as `additional_headers`.

Every later PR on the Go side built on the unverified parity claim instead of
checking it against Python:

- kagent-dev#1858 added `KAGENT_PROPAGATE_TOKEN` support for Go, on top of the same
  static-wins rule.
- kagent-dev#1880 added the STS `headerProvider` tier, and added
  `TestStaticHeaders_OverrideDynamic` to lock the (incorrect) rule in.
- kagent-dev#2044 and kagent-dev#2087 both tried to invert it for this exact
  discovery-credential-vs-OBO conflict, gated behind a new opt-in env var
  (`KAGENT_PROPAGATE_TOKEN_OVERRIDES_STATIC`) plus, in kagent-dev#2087, `X-Actor-Token`
  forwarding for the displaced static credential. kagent-dev#2044 cited
  `TestStaticHeaders_OverrideDynamic` as proof the behaviour was "intentional
  today". kagent-dev#2087 was self-closed by its author: "I think this is the bad
  direction."
- kagent-dev#2071, kagent-dev#2073, kagent-dev#2108 tried to design the credential model properly and were
  all closed "until API v2 lands".

None of that machinery was necessary: the behaviour being preserved as
"intentional" was a bug from the day it was written. This change makes Go
match what Python has done all along — no new env var, no actor-token
forwarding, no opt-in.

## Change

`headerRoundTripper.RoundTrip` now applies static headers *first*, as
defaults. `propagateToken`, `allowedHeaders`, and `headerProvider` are applied
after, in their existing relative order, and override a static header of the
same name on collision.

A header is only ever overridden when it is explicitly opted into forwarding
via one of those three mechanisms — an incoming request cannot clobber a
static header just by sending one with a matching name
(`TestUnlistedRequestHeader_DoesNotOverrideStatic`).

Related: kagent-dev#1679, kagent-dev#1733, kagent-dev#1858, kagent-dev#1880, kagent-dev#2044, kagent-dev#2087, kagent-dev#2071, kagent-dev#2073, kagent-dev#2108

Signed-off-by: Brian Fox <878612+onematchfox@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 25, 2026 14:33
}
resp.Body.Close()

if capturedAuth != "Bearer static" {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion was wrong to begin with. Header round-tripper is configured above with allowedHeaders: []string{"Authorization"}, indicating that the Authorization header (the user's token) should be forwarded.

@onematchfox

Copy link
Copy Markdown
Contributor Author

Only opening against release/v0.10.x due to ongoing work on V2 and mention in linked PRs of changes to this area. Let me know if I should cherry-pick to main as well.

@github-actions github-actions Bot added the bug Something isn't working label Aug 25, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Go ADK’s MCP HTTP header precedence to match the Python runtime: static headers configured on the MCP server now act as defaults, while per-request/dynamic headers (propagated token, allowedHeaders forwarding, and STS/headerProvider) can override them. This resolves the gateway topology issue where a static discovery credential was unintentionally clobbering end-user Authorization on tool calls in the Go runtime.

Changes:

  • Apply static MCP server headers first in headerRoundTripper.RoundTrip, allowing dynamic/per-call headers to override on collision.
  • Update header precedence documentation in registry.go to reflect the new ordering.
  • Adjust and extend unit tests around precedence/override gating for static vs forwarded vs dynamic headers.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
go/adk/pkg/mcp/registry.go Reorders header application so static headers become defaults and dynamic headers can override, aligning Go behavior with Python MCP header merge precedence.
go/adk/pkg/mcp/registry_test.go Updates/extends tests to cover the new precedence behavior and gating semantics for when incoming headers may override static defaults.
Suppressed comments (2)

go/adk/pkg/mcp/registry_test.go:153

  • This assertion is intended to validate that an allowed request header overrides a static default, but because both values are currently identical placeholders, the test can't detect incorrect precedence. Use distinct static vs incoming values and assert the incoming value.
	if capturedAuth != "Bearer incoming" {
		t.Errorf("Authorization: got %q, want %q", capturedAuth, "Bearer incoming")
	}

go/adk/pkg/mcp/registry_test.go:610

  • This assertion is meant to verify headerProvider-sourced headers override static defaults, but the test currently sets both static and dynamic Authorization values to identical placeholders, so it can’t detect precedence regressions. Use distinct values and assert on the dynamic one.
	if capturedAuth != "Bearer dynamic" {
		t.Errorf("Authorization: got %q, want %q", capturedAuth, "Bearer dynamic")
	}

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +98 to +118
ctx := a2aCtx(map[string][]string{
"Authorization": {"Bearer incoming"},
})

rt := &headerRoundTripper{
base: newTestTransport(t),
headers: map[string]string{"Authorization": "Bearer static"},
// Deliberately no allowedHeaders, no propagateToken, no headerProvider:
// nothing opts Authorization into being forwarded from the request.
}

req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL, nil)
resp, err := rt.RoundTrip(req)
if err != nil {
t.Fatalf("RoundTrip failed: %v", err)
}
resp.Body.Close()

if capturedAuth != "Bearer static" {
t.Errorf("Authorization: got %q, want %q", capturedAuth, "Bearer static")
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants