[BUG] BYO agents crashloop on 0.10.x: controller mounts a model-less declarative config at /config

[rtemperini]

🎯 Affected Service(s)

Controller Service

🚦 Impact/Severity

Blocker (for BYO agents on 0.10.x)

🐛 Bug Description

On 0.10.x, BYO agents crashloop on startup because the controller mounts a config
intended for the declarative runtime into them. The config it renders has no model,
and the runtime schema requires one.

There are two defects here, one behavioural and one observability:

1. The controller renders an unusable config for BYO agents and mounts it. BYO
   agents receive /config/config.json containing {"model":null,"description":"…","instruction":""}.
   Any BYO runtime that loads that file on startup fails validation.
2. It does so silently. Rendering the config succeeds, so the controller logs
   nothing and reports no condition. The only signal is the agent pod's crash loop,
   which points at the agent image rather than at the rendered config.

Worth stating plainly, because it explains how this got missed: the two
implementations of this one schema disagree about whether a model-less config is
legal. The Go side was taught to accept it; the Python side that actually consumes
the file was not.

🔄 Steps To Reproduce

Apply any BYO agent:

apiVersion: kagent.dev/v1alpha2
kind: Agent
metadata:
  name: byo-agent
  namespace: default
spec:
  type: BYO
  description: A BYO test agent
  byo:
    deployment:
      image: example.com/my-agent:latest

Then read back the generated Secret:

$ kubectl get secret byo-agent -n default -o jsonpath='{.data.config\.json}' | base64 -d
{"model":null,"description":"A BYO test agent","instruction":""}

and confirm it is mounted:

$ kubectl get deploy byo-agent -n default \
    -o jsonpath='{.spec.template.spec.containers[0].volumeMounts}'
[{"mountPath":"/config","name":"config"}, ...]

This is also visible without a cluster — it is committed as a golden fixture. At
v0.10.0-rc3, testdata/outputs/byo_agent.json
records both the "model": null config and the /config mount as expected output.

🤔 Expected Behavior

A BYO agent should not be given the declarative runtime's config. Before the change
below, it wasn't: cfg stayed nil for BYO, so config.json was empty and no config
volume was mounted.

📱 Actual Behavior

The BYO runtime reads /config/config.json and fails validation:

pydantic_core._pydantic_core.ValidationError: 1 validation error for AgentConfig
model
  Input should be a valid dictionary or object to extract fields from
  [type=model_attributes_type, input_value=None, input_type=NoneType]

🔍 Additional Context

Mechanism (line numbers at 03bec863, current release/v0.10.x):

1. compiler.go:139-149 —
   the BYO branch resolves the deployment and now also sets a minimal config:

   case v1alpha2.AgentType_BYO:
       dep, err = resolveByoDeployment(agent)
       // BYO currently does not share configuration with the declarative
       // runtime so this is a minimal config to support propagating agent config
       // to BYO agents through this format
       cfg = &adk.AgentConfig{
           Description: spec.Description,
       }

2. manifest_builder.go:205
   and :233 —
   buildConfigSecret gates the rendered config and the /config volume on a bare
   if cfg != nil, with no BYO/declarative distinction. Now that cfg is always
   non-nil, BYO agents get both.

3. go/api/adk/types.go:587 —
   Model is an interface tagged json:"model" with no omitempty, so the zero
   value marshals to null.

4. python/.../adk/types.py:391 —
   model: ModelUnion = Field(discriminator="type") is required with no default.
   (Contrast summarizer_model one line above at :360, which is explicitly
   | None = Field(default=None, …).)

Regression point: b04769e8 — "feat(sandbox-agents): Store session state in
durableDir volume for declarative sandbox agents" (#2171).

Affected versions:

$ git tag --contains b04769e8
v0.10.0-beta7 … v0.10.0-beta11, v0.10.0-rc1, v0.10.0-rc2, v0.10.0-rc3

First affected: v0.10.0-beta7. Last clean 0.10.x: v0.10.0-beta6. No
v0.9* tag contains it, so 0.9.x is unaffected.

On the Go/Python divergence. #2171 was scoped to declarative sandbox agents, and
its own description notes that session storage for BYO was explicitly not supported —
so the BYO config assignment was outside the change's intent. The same commit did
account for the null model on the Go side, adding this to AgentConfig.UnmarshalJSON:

// BYO agents carry a minimal config with no model (it marshals as "model":null); a config
// without a model is legal and must round-trip — ParseModel would reject it.

So a model-less config is deliberately legal in Go. It is not legal in the Python
runtime that reads the mounted file, and that side was not updated to match. That
asymmetry is the actual bug; it's an easy one to miss, since the Go change makes the
round-trip tests pass.

Please don't fix this with omitempty alone. Adding omitempty to
AgentConfig.Model looks like a one-line fix, but it does not stop the crash. The
config.json file still exists and is still mounted; it just loses the model key,
and since the Python field is required with no default, validation still fails —
only the error changes. Measured against the schema as declared at rc3:

Rendered config.json	pydantic result
{"model":null,"description":"…","instruction":""} (today)	type='model_attributes_type', "Input should be a valid dictionary or object to extract fields from"
{"description":"…","instruction":""} (with omitempty)	type='missing', "Field required"

The fix has to be controller-side — either leave cfg nil for BYO, or gate the
config Secret and volume on agent type. Note that leaving cfg nil is not a clean
revert: compiler.go:160
dereferences cfg.SessionDBURL unguarded when the agent runs in sandbox workload
mode, so a nil cfg would panic there. Gating in buildConfigSecret also matches
two BYO carve-outs already in that file (needsSRTSettings, and the service
appProtocol marker). omitempty is still worth doing as defence in depth.

Please fix on release/v0.10.x, not just main. main won't help anyone here:
26732e86 ("chore: remove legacy ACP and controller runtime", #2565) deleted
go/core/internal/controller/translator/ outright, git tag --contains 26732e86
returns nothing, and that commit is not on release/v0.10.x. On main there is no
Agent reconciler at all — a BYO Agent isn't reconciled there. Meanwhile
v0.10.0-rc3 is the newest tag and the newest image on GHCR, with no 0.10.0 final
and no rc4, so rc3 is what people are actually running. release/v0.10.x has
unreleased commits past rc3, none of which touch this code
(git diff v0.10.0-rc3 origin/release/v0.10.x -- go/core/internal/controller/translator/agent/
is empty). It would be good not to ship 0.10.0 with this.

Possibly related, not duplicates:

• RemoteMCPServer headersFrom secret is baked into agent config and never re-rendered on rotation — agents break permanently with short-lived tokens #2404 — open, and the closest neighbour: also about the rendered agent config Secret
  holding content that is wrong at runtime, where the resulting failure isn't surfaced
  by the controller. Different trigger, but defect 2 above may share a root cause.
• [FEATURE] Support optional modelConfig reference in BYO Agent CRD (spec.type: BYO) for secure, centralized model credential governance #1367 — closed as stale; a request to make modelConfig optional for BYO agents.
  Different ask, but it rests on the same premise that BYO agents need no model.

Nothing asserts that a BYO agent should not receive the config, which is why this
landed quietly: the golden fixture simply recorded the "model": null config and the
/config mount as the expected output, so the suite stayed green.

🙋 Are you willing to contribute?

Yes — PR against release/v0.10.x to follow, with a regression test.