Skip to content

[bot] RubyLLM moderation API (RubyLLM.moderate) not instrumented #176

@braintrust-bot

Description

@braintrust-bot

Summary

RubyLLM.moderate is a model execution call that invokes OpenAI's moderation API to classify text for harmful content. It is not instrumented by the Braintrust Ruby SDK. The existing RubyLLM integration only patches RubyLLM::Chat.

This is distinct from issue #136 (RubyLLM embed, paint, and transcribe APIs not instrumented), which covers three other non-chat RubyLLM execution surfaces but explicitly omits moderation.

What is missing

RubyLLM.moderate("text") (aliased as RubyLLM::Moderation.moderate) makes a live API request to OpenAI's moderation endpoint using models such as omni-moderation-latest or text-moderation-007. It returns a RubyLLM::Moderation object with:

  • flagged — overall boolean
  • Per-category flags and confidence scores
  • id and model from the API response

This is the same class of execution call as the moderations already instrumented for the openai and ruby-openai integrations (ModerationsPatcher in both). A parallel RubyLLM::ModerationPatcher would follow the same pattern.

What a span should capture

  • Input: the text string being moderated
  • Metadata: model (e.g., omni-moderation-latest), provider (ruby_llm), endpoint
  • Output: flagged categories and scores (or summary)
  • Metrics: no token usage is returned by the moderation API, but latency should be captured

Braintrust docs status

not_found — The Braintrust RubyLLM integration docs at https://www.braintrust.dev/docs/integrations/sdk-integrations/ruby-llm cover chat completions and tool calls but do not mention moderation. The OpenAI provider docs at https://www.braintrust.dev/docs/providers/openai document moderation instrumentation via the openai gem but not via RubyLLM.

Upstream sources

Local files inspected

  • lib/braintrust/contrib/ruby_llm/patcher.rb — defines only ChatPatcher; no ModerationPatcher
  • lib/braintrust/contrib/ruby_llm/integration.rb — registers only [ChatPatcher]; loaded? checks defined?(::RubyLLM::Chat) only
  • lib/braintrust/contrib/openai/instrumentation/moderations.rb — demonstrates the existing moderation instrumentation pattern in this repo
  • lib/braintrust/contrib/openai/patcher.rb (lines 131–170) — ModerationsPatcher for reference

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions