Skip to content

fix(runtime): surface agent_call middleware bypass on sync agent methods - #115

Open
Atharva-Kanherkar wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
Atharva-Kanherkar:fix/sync-agent-call-middleware-gap
Open

fix(runtime): surface agent_call middleware bypass on sync agent methods#115
Atharva-Kanherkar wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
Atharva-Kanherkar:fix/sync-agent-call-middleware-gap

Conversation

@Atharva-Kanherkar

@Atharva-Kanherkar Atharva-Kanherkar commented Aug 8, 2026

Copy link
Copy Markdown

refs #114

agent_call middleware is documented as wrapping the entire agent method, but it only wraps traced async methods. sync methods, @no_trace methods (sync or async), staticmethod/classmethod, and methods inherited from non-Agent bases all run outside it, so a registered guard silently does not apply.

no behaviour change. this warns instead of staying silent, and fixes the docs that promised full coverage.

real gpt-5-mini run of the reproducer from #114, on this branch:

repro_114_real.py:45: RuntimeWarning: agent_call middleware is registered, but it
does not apply to these methods on PaymentAgent: charge_card. Middleware is async
and only wraps async agent methods that the metaclass instruments, so synchronous
methods, @no_trace methods, staticmethod/classmethod, and methods inherited from
non-Agent bases all execute outside it. Guards that block, authenticate, or
rate-limit will not run for them - including when generated CodeAct Python calls
them. Declare such a capability as a traced 'async def' method to bring it under
middleware, or enforce the policy inside the method body.
  result = await agent.perform()

{'result': 'done', 'charges': [100], 'intercepted_methods': ['perform']}

same result as the issue, but the gap is now reported at await agent.perform(), before the model reaches charge_card.

changes:

  • warn once per class, naming every uncovered method, when middleware is registered
  • correct the docs in middleware.py, event_manager.py, the middleware skill, and the relay example
  • tests for the asymmetry, each uncovered kind, both delivery paths, and -W error

6263 passed, 152 skipped. tests/test_mcp excluded, optional dep not installed here. ruff and format clean.

open question: should this raise instead of warn? and @no_trace async bypassing middleware may be worth its own issue, since one decorator is controlling both tracing and policy.

`agent_call` middleware is documented as wrapping the "entire agent method
execution" and as the auth / rate-limiting layer, but the sync method wrapper
skips it — middleware is async and cannot wrap a sync calling convention.

A guard registered via `intercept("agent_call", ...)` therefore appears
installed while a synchronous capability runs unchecked, including when
generated CodeAct Python calls it. The failure is silent, and `def` vs
`async def` is not a keyword anyone reads as a policy boundary.

This does not change the execution model. It makes the gap visible and
corrects the documentation that promised coverage sync methods never had:

- Emit a RuntimeWarning, once per wrapped method, when a sync agent method
  runs while agent_call middleware is registered. Calls originating inside a
  CodeAct cell route to the logger instead: cells redirect sys.stderr into a
  capture buffer, so a warning there would be invisible to the developer and
  would be fed back to the model as cell output.
- Correct the AgentCallContext and intercept() docstrings.
- Add regression tests covering the sync/async asymmetry, the delivery
  channel in both call paths, and the once-per-method dedup.

Refs NVIDIA-NeMo#114

Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
Adversarial review of the previous commit found the warning did not actually
reach developers, and covered only one of the several ways a method can fall
outside agent_call middleware.

Delivery — the warning was routed to `logger.warning` when it originated inside
a CodeAct cell, on the reasoning that cells redirect sys.stderr into a capture
buffer fed back to the model. But `nooa` attaches a NullHandler to its root
logger, so that handler consumed the record instead of letting it fall through
to logging.lastResort: with no logging configuration, nothing was emitted on
either stream. The test used caplog, which attaches its own handler and hid it.

Now always `warnings.warn`, clearing the stderr buffer contextvar for the
duration of the call so ContextVarStream falls through to the real stream. The
warning reaches the developer, the cell output the model reads stays clean, and
warning filters apply uniformly.

Coverage — the diagnostic lived in the sync wrapper, so it could only fire for
methods that wrapper is built for. @no_trace methods (sync and async),
staticmethod/classmethod, and methods inherited from non-Agent bases are never
wrapped at all, and all bypass middleware silently. Note this includes async
methods: @no_trace async is uninstrumented and therefore also unguarded.

Added a class scan that runs from the async wrapper at the entry point, keyed on
the attribute rather than on how it was declared: covered iff the metaclass
wrapped it and the result is a coroutine function. This reports every uncovered
method proactively, before the model can call one, and reports nooa-internal
methods to nobody. The per-call sync warning remains for code that calls a sync
capability without entering an async method at all.

Error promotion — the emit path was inside a blanket `except Exception`, so
`-W error` was swallowed, and the delivered flag was set before the emit, so a
warning that raised was never retried. The flag is now set after a successful
emit and only diagnostic construction is guarded.

Docs — corrected the remaining whole-method coverage claims in the middleware
module docstring, the middleware-hooks skill, and the NeMo Relay example.

Refs NVIDIA-NeMo#114

Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
@Atharva-Kanherkar
Atharva-Kanherkar marked this pull request as ready for review August 8, 2026 09:29
@furgalep furgalep self-assigned this Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants