Skip to content

[bot] dspy gem (dspy.rb) not instrumented #180

@braintrust-bot

Description

@braintrust-bot

Summary

The dspy gem (dspy.rb, ~70K downloads, v1.0.0, April 2026) is the Ruby port of the DSPy LLM programming framework and is not instrumented by this SDK. Braintrust documents DSPy integration for Python, capturing module-level spans for Predict, ChainOfThought, and other DSPy modules — but there is no equivalent Ruby instrumentation for dspy.rb.

What is missing

No instrumentation exists for any dspy execution surface. Key APIs that should be instrumented:

Module Execution

  • DSPy::Predict#call(**inputs) — single-pass LLM call with typed input/output contract. The core building block of all DSPy programs.
  • DSPy::ChainOfThought#call(**inputs) — LLM call that surfaces intermediate chain-of-thought reasoning steps before the final answer. Produces a richer trace than Predict due to the reasoning span.
  • DSPy::ReAct#call(**inputs) — agentic reasoning loop that interleaves LLM calls with tool executions. May span multiple LLM turns and tool invocations.

Tool Execution (ReAct)

  • Tool calls dispatched within DSPy::ReAct loops should be captured as child spans, including tool name, inputs, and output.

LM Configuration

  • DSPy::LM.new('openai/gpt-4o-mini', ...) — LM adapter configuration, useful for capturing which model and provider is driving each module.

Expected instrumentation

Module execution spans should capture:

  • Module type (Predict, ChainOfThought, ReAct)
  • Signature type name (typed input/output schema)
  • Input fields and values
  • LLM calls made (model, provider, tokens)
  • Intermediate reasoning steps (for ChainOfThought)
  • Tool calls and results (for ReAct)
  • Typed output fields

Braintrust docs status

unclear — Braintrust documents DSPy integration at https://www.braintrust.dev/docs/integrations/sdk-integrations/dspy and captures Predict, ChainOfThought, and other DSPy module spans with full tracing. However, the documented integration requires dspy>=2.6.0 (Python) and covers only Python. No Ruby-specific dspy.rb instrumentation is documented.

Upstream sources

Local repo files inspected

  • lib/braintrust/contrib.rb (lines 195–205) — registers only 4 integrations: OpenAI, RubyOpenAI, RubyLLM, Anthropic. No dspy integration.
  • lib/braintrust/contrib/ — no directory or files for dspy
  • Appraisals — no appraisal scenarios for dspy
  • braintrust.gemspec — no mention of dspy
  • Grep for dspy, DSPy, Predict, ChainOfThought, ReAct across entire codebase returns zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions