Summary
The ai-agents gem (~718K downloads, v0.11.0, May 2026) is an actively maintained Ruby agent execution framework that is not instrumented by this SDK. While it is built on top of ruby_llm (which is already instrumented for LLM calls), ai-agents adds its own agent orchestration layer — multi-agent handoffs, tool execution loops, and runner coordination — that is invisible to the existing ruby_llm integration.
Braintrust's documented Ruby LLM integration explicitly covers "direct RubyLLM usage patterns" (chat completions, tool calls, token usage) and does not address agent-level execution spans added by frameworks on top of ruby_llm.
What is missing
No instrumentation exists for any ai-agents execution surface. Key APIs that should be instrumented:
Agent Run
Agents::Runner.with_agents(agent1, ...).run("message", context:) — the main agent execution entry point. Returns a result with output and updated context. Each call may span multiple LLM turns, tool executions, and agent handoffs.
Tool Execution
Agents::Tool#perform(tool_context, **params) — custom tool execution. Tools are dispatched automatically by the agent during conversation based on LLM decisions. These calls should be captured as child spans within an agent run.
Multi-Agent Handoffs
- Agent-to-agent handoffs during a conversation turn should be captured as spans showing which agent handled which portion of the request.
Expected instrumentation
Agent run spans should capture:
- Input message and context
- Which agent(s) handled the request (including handoffs)
- Tool calls made and their results
- Number of LLM turns taken
- Final output
Braintrust docs status
not_found — ai-agents is not listed in Braintrust's integrations directory at https://www.braintrust.dev/docs/integrations. The Ruby LLM integration page at https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm explicitly focuses on direct ruby_llm usage and does not address agent frameworks built on top of it.
Upstream sources
Local repo files inspected
lib/braintrust/contrib.rb (lines 195–205) — registers only 4 integrations: OpenAI, RubyOpenAI, RubyLLM, Anthropic. No ai-agents integration.lib/braintrust/contrib/ — no directory or files for ai_agents or ai-agentsAppraisals — no appraisal scenarios for ai-agentsbraintrust.gemspec — no mention of ai-agents- Grep for
ai.agents, ai_agents, Agents::Runner across entire codebase returns zero matches
Summary
The
ai-agentsgem (~718K downloads, v0.11.0, May 2026) is an actively maintained Ruby agent execution framework that is not instrumented by this SDK. While it is built on top ofruby_llm(which is already instrumented for LLM calls),ai-agentsadds its own agent orchestration layer — multi-agent handoffs, tool execution loops, and runner coordination — that is invisible to the existingruby_llmintegration.Braintrust's documented Ruby LLM integration explicitly covers "direct RubyLLM usage patterns" (chat completions, tool calls, token usage) and does not address agent-level execution spans added by frameworks on top of
ruby_llm.What is missing
No instrumentation exists for any
ai-agentsexecution surface. Key APIs that should be instrumented:Agent Run
Agents::Runner.with_agents(agent1, ...).run("message", context:)— the main agent execution entry point. Returns a result with output and updated context. Each call may span multiple LLM turns, tool executions, and agent handoffs.Tool Execution
Agents::Tool#perform(tool_context, **params)— custom tool execution. Tools are dispatched automatically by the agent during conversation based on LLM decisions. These calls should be captured as child spans within an agent run.Multi-Agent Handoffs
Expected instrumentation
Agent run spans should capture:
Braintrust docs status
not_found—ai-agentsis not listed in Braintrust's integrations directory at https://www.braintrust.dev/docs/integrations. The Ruby LLM integration page at https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm explicitly focuses on directruby_llmusage and does not address agent frameworks built on top of it.Upstream sources
Local repo files inspected
lib/braintrust/contrib.rb(lines 195–205) — registers only 4 integrations: OpenAI, RubyOpenAI, RubyLLM, Anthropic. Noai-agentsintegration.lib/braintrust/contrib/— no directory or files forai_agentsorai-agentsAppraisals— no appraisal scenarios forai-agentsbraintrust.gemspec— no mention ofai-agentsai.agents,ai_agents,Agents::Runneracross entire codebase returns zero matches