fix(agents): close the inner run_async when an agent node stops early - #7202
Open
AtulJoshi1206 wants to merge 1 commit into
Open
AtulJoshi1206 wants to merge 1 commit into
AtulJoshi1206 wants to merge 1 commit into
Conversation
BaseAgent._run_impl iterated run_async with a bare async for, so when BaseNode.run's Aclosing closed it early the inner generator was dropped rather than closed. Its cleanup then fell to the asyncgen finalizer hook, which resumes it in a different contextvars context, and the OTel span it is suspended in fails to detach: "ValueError: Token was created in a different Context". after_agent_callback and the exit-stack teardown ran out of band with it. Wrap it in Aclosing, which this module already imports and which both BaseNode.run and the LlmAgent._run_impl override already use.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
No existing issue; described below following the bug-report structure.
Describe the Bug:
BaseAgent._run_impl, the node-mode entry point, iteratesrun_asyncwithout closing it:Every other consumer of an agent generator in the repo wraps it in
Aclosing, includingBaseNode.run, which is this method's only caller, andLlmAgent._run_impl, the override of this very method.Aclosingis already imported inbase_agent.py.When
BaseNode.run'sAclosingcloses_run_implearly,GeneratorExitis raised at theyield, the frame unwinds, and the innerrun_asyncgenerator is dropped rather than closed. That generator is suspended insiderecord_agent_invocation, which holds an OpenTelemetry span, and inside thetry/except asyncio.CancelledErrorthat runsafter_agent_callback.Impact:
Its cleanup is deferred to the async-generator finalizer hook, which resumes it as a separate task in a different
contextvarscontext. That is not merely late, it fails:So an early-terminated agent node leaves its OTel span un-detached and runs its
after_agent_callbackand exit-stack teardown out of band. Early termination is ordinary: a workflow finishing, a node being interrupted, or an error upstream.Steps to Reproduce:
On current
main(665ec9835), consume one event from_run_impland close it:Observed Behavior:
plus the
Failed to detach contexttraceback above on stderr. The agent'sfinallyruns only later, from the finalizer hook.Expected Behavior:
aclose()returns only after the inner generator has been closed, so cleanup ordering is deterministic and the OTel span is detached in the context that attached it.Solution:
Wrap the inner generator in the
Aclosinghelper already imported in this module, matchingBaseNode.runandLlmAgent._run_impl.Testing Plan
Unit Tests:
Added
test_run_impl_closes_run_async_when_consumer_stops_early, which asserts the inner generator'sfinallyhas run by the timeaclose()returns. Verified it fails on an unmodifiedbase_agent.pywithassert [] == [True]and passes with this change.Manual End-to-End (E2E) Tests:
Driven through
_run_impldirectly, since this is the node-mode adapter. Consuming one event from a three-event agent and closing:The ordering flips and the
Failed to detach contexttraceback disappears.Checklist
Additional context
No public API change, and the fully-consumed path is unchanged.
Two sibling adapters have the same bare
async foroverrun_asyncand are deliberately left out of this PR to keep it to one file:agents/_managed_agent.pyandlabs/antigravity/_antigravity_agent.py, whose comment already says "Keep in sync with BaseAgent._run_impl". Happy to follow up on both, or fold them in here if you would rather have one change.🤖 Generated with Claude Code