feat: add Braintrust tracing integration

[dpguthrie]

Summary

Adds native Braintrust observability support to Langflow, following the same BaseTracer pattern used by existing monitoring integrations (Langfuse, LangSmith, Opik, Arize Phoenix, LangWatch, Traceloop).

• New tracer: src/backend/base/langflow/services/tracing/braintrust.py implementing BaseTracer
• Service registration: Added _get_braintrust_tracer() and _initialize_braintrust_tracer() to service.py
• Documentation: New docs/docs/Develop/integrations-braintrust.mdx page with setup instructions
• Sidebar: Added Braintrust to the Monitoring section in docs/sidebars.js

How it works

The integration activates automatically when the BRAINTRUST_API_KEY environment variable is set (same pattern as all other monitoring integrations). It provides:

1. Component-level tracing -- each Langflow component execution creates a span in Braintrust
2. Deep LangChain tracing -- returns an inline LangChain callback handler from get_langchain_callback() that captures LLM calls with token metrics, time-to-first-token, tool usage, retriever queries, and chain execution details

Dependencies

• braintrust -- the only required dependency. The tracer includes an inline LangChain callback handler built entirely on the braintrust SDK primitives (start_span, log, end), so no additional packages like braintrust-langchain are needed.

Environment variables

Variable	Required	Default	Description
BRAINTRUST_API_KEY	Yes	--	Braintrust API key
BRAINTRUST_API_URL	No	https://api.braintrust.dev	API endpoint
BRAINTRUST_PROJECT	No	Langflow project name	Project name in Braintrust

Test plan

• Set BRAINTRUST_API_KEY, start Langflow, run a flow, verify traces appear in Braintrust dashboard
• Verify tracer auto-disables when BRAINTRUST_API_KEY is not set (no errors, no impact)
• Verify component-level spans are created for each node in the flow
• Verify LLM calls show token metrics and time-to-first-token
• Verify existing monitoring integrations (Langfuse, LangSmith, etc.) are unaffected

Summary by CodeRabbit

• New Features

  • Integrated Braintrust as a new observability platform, enabling comprehensive real-time tracing of Langflow flows and components with support for nested spans and token-level metrics.

• Documentation

  • Added Braintrust integration guide covering account setup, API key configuration, environment variables, cross-platform installation, trace collection workflow, and dashboard usage.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

Walkthrough

This PR adds Braintrust integration to Langflow for capturing and forwarding component and flow traces. It includes documentation, sidebar navigation updates, a new BraintrustTracer implementation with environment-based configuration, and integration into the existing tracing service infrastructure.

Changes

Cohort / File(s)	Summary
Documentation & Navigation docs/docs/Develop/integrations-braintrust.mdx, docs/sidebars.js	New Braintrust integration guide covering prerequisites, environment configuration, platform-specific setup commands, trace collection behavior, and dashboard usage. Sidebar entry added under Observability > Monitoring.
BraintrustTracer Implementation src/backend/base/langflow/services/tracing/braintrust.py	New tracer class implementing BaseTracer with lifecycle methods (add_trace, end_trace, end) to emit per-flow and per-component spans to Braintrust. Handles environment-based API key/URL/project configuration, gracefully manages braintrust import failures, recursively converts Langflow-specific types to JSON-serializable forms, and tracks root/child spans with metadata.
Tracing Service Integration src/backend/base/langflow/services/tracing/service.py	Added BraintrustTracer factory function and initialization method invoked in start_tracers pipeline to instantiate and register the braintrust tracer with trace context.

Sequence Diagram

sequenceDiagram
    participant LF as Langflow Flow
    participant TS as TracingService
    participant BT as BraintrustTracer
    participant BR as Braintrust Backend

    LF->>TS: start_tracers()
    TS->>BT: _initialize_braintrust_tracer()
    BT->>BT: _get_config() from env
    BT->>BR: Initialize root span (flow metadata)
    BR-->>BT: Root span created

    LF->>BT: add_trace(component inputs)
    BT->>BR: Create child span
    BR-->>BT: Child span stored

    LF->>BT: end_trace(component outputs/logs)
    BT->>BR: Finish child span with outputs/logs
    BR-->>BT: Span finalized

    LF->>BT: end(flow outputs/error)
    BT->>BR: Finish root span with flow I/O
    BR-->>BT: Root span closed

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

---

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 3 warnings)

Check name	Status	Explanation	Resolution
Test Coverage For New Implementations	❌ Error	PR introduces BraintrustTracer implementation but lacks corresponding test files for validating the new functionality.	Add comprehensive tests covering initialization, span creation, error handling, type conversion, and integration with TracingService.
Docstring Coverage	⚠️ Warning	Docstring coverage is 21.43% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Quality And Coverage	⚠️ Warning	The PR introduces 240-line BraintrustTracer implementation with zero dedicated unit tests and missing Braintrust tracer verification in existing test suite.	Create dedicated test file with comprehensive unit tests for configuration, setup, span management, data conversion, and update existing tests to include Braintrust tracer verification.
Test File Naming And Structure	⚠️ Warning	BraintrustTracer implementation lacks test coverage in existing test infrastructure and requires dedicated test file.	Add _get_braintrust_tracer to mock_tracers fixture, add braintrust assertion to test_start_end_tracers, and create comprehensive test_braintrust_tracer.py.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat: add Braintrust tracing integration' directly and accurately summarizes the main change: adding a new Braintrust tracing integration to Langflow. It is concise, specific, and clearly communicates the primary objective.
Excessive Mock Usage Warning	✅ Passed	No test files are included in this PR, so there are no mocks to assess for excessive use.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🤖 Fix all issues with AI agents

In `@docs/docs/Develop/integrations-braintrust.mdx`:
- Around line 1-4: The MDX frontmatter is missing required fields; update the
YAML frontmatter for the Braintrust doc by adding the missing `description` and
`sidebar_position` keys alongside the existing `title` (e.g., set a short
descriptive string for `description` and an integer for `sidebar_position`) so
the frontmatter contains `title`, `description`, and `sidebar_position`.
- Around line 31-33: Several bash code blocks (instances of the triple-backtick
fences that start with "```bash") are missing the required title attribute;
update each code fence (e.g., the ones currently written as "```bash") to
include a title like title="Terminal" (so they read e.g. ```bash
title="Terminal") or another descriptive title per block; ensure you change
every bash block in the file (including the blocks noted around the earlier
examples) so each specifies both the language and a title attribute.

In `@src/backend/base/langflow/services/tracing/braintrust.py`:
- Around line 50-68: The Braintrust tracer constructor (__init__ in
braintrust.py) accepts user_id and session_id but _initialize_braintrust_tracer
in service.py does not forward them, so root span metadata always has None;
update _initialize_braintrust_tracer to accept (or extract) the service-layer
user_id and session_id and pass them into the Braintrust tracer constructor when
instantiating the tracer (same pattern used by langfuse/opik tracers) so the
root span metadata includes the correct user_id and session_id.

In `@src/backend/base/langflow/services/tracing/service.py`:
- Around line 229-238: The Braintrust tracer is not receiving user_id and
session_id from the TraceContext, so root span metadata will always be None;
update _initialize_braintrust_tracer to pass trace_context.user_id and
trace_context.session_id into the braintrust_tracer call (alongside trace_name,
trace_type, project_name, trace_id) so the tracer's constructor receives those
values (TraceContext and braintrust_tracer are the relevant symbols to update).

🧹 Nitpick comments (3)

src/backend/base/langflow/services/tracing/braintrust.py (3)

104-106: logger.exception is too noisy for a missing optional dependency.

logger.exception logs the full stack trace. For an expected condition (optional braintrust package not installed), a simple logger.info or logger.warning with the message is more appropriate. Other tracers in the codebase handle ImportError more quietly.

Proposed fix

         except ImportError:
-            logger.exception("Could not import braintrust. Please install it with `pip install braintrust`.")
+            logger.info("Could not import braintrust. Please install it with `pip install braintrust`.")
             return False

---

212-213: Redundant isinstance check — HumanMessage and SystemMessage are subclasses of BaseMessage.

Checking isinstance(value, (BaseMessage, HumanMessage, SystemMessage)) is equivalent to isinstance(value, BaseMessage) since the latter two inherit from it. The extra types can be removed for clarity.

Proposed fix

-        if isinstance(value, (BaseMessage, HumanMessage, SystemMessage)):
+        if isinstance(value, BaseMessage):
             return value.content

---

208-217: Message check must precede BaseMessage — and str(None) produces "None" string.

Two observations on the ordering and behavior:

1. Order is correct — Message is checked before BaseMessage, which is important since Message may share an ancestor. Good.

2. types.NoneType in the tuple at line 216 converts None to the string "None" rather than passing through as None. If this is intentional, a brief comment would help; otherwise the NoneType check can be dropped so None passes through as-is from return value.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Frontmatter is missing description and sidebar_position.

As per coding guidelines, YAML frontmatter must include title, description, and sidebar_position.

Proposed fix

 ---
 title: Braintrust
 slug: /integrations-braintrust
+description: Configure Langflow to send tracing data to Braintrust for debugging and analyzing flow executions.
+sidebar_position: 1
 ---

As per coding guidelines: "Markdown files must include YAML frontmatter with title, description, and sidebar_position".

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	---
	title: Braintrust
	slug: /integrations-braintrust
	---
	---
	title: Braintrust
	slug: /integrations-braintrust
	description: Configure Langflow to send tracing data to Braintrust for debugging and analyzing flow executions.
	sidebar_position: 1
	---

🤖 Prompt for AI Agents

In `@docs/docs/Develop/integrations-braintrust.mdx` around lines 1 - 4, The MDX
frontmatter is missing required fields; update the YAML frontmatter for the
Braintrust doc by adding the missing `description` and `sidebar_position` keys
alongside the existing `title` (e.g., set a short descriptive string for
`description` and an integer for `sidebar_position`) so the frontmatter contains
`title`, `description`, and `sidebar_position`.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Code blocks are missing the title attribute.

Per coding guidelines, code blocks must include a title attribute and specify the language (e.g., ```bash title="Terminal"). All bash code blocks in this file are missing title.

As per coding guidelines: "Code blocks must include a title attribute and specify the language".

Also applies to: 38-40, 52-58, 63-65, 75-77

🤖 Prompt for AI Agents

In `@docs/docs/Develop/integrations-braintrust.mdx` around lines 31 - 33, Several
bash code blocks (instances of the triple-backtick fences that start with
"```bash") are missing the required title attribute; update each code fence
(e.g., the ones currently written as "```bash") to include a title like
title="Terminal" (so they read e.g. ```bash title="Terminal") or another
descriptive title per block; ensure you change every bash block in the file
(including the blocks noted around the earlier examples) so each specifies both
the language and a title attribute.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Constructor does not accept user_id / session_id from the service layer.

The __init__ signature accepts user_id and session_id, and stores them in metadata, but _initialize_braintrust_tracer in service.py (line 233) does not pass them. Compare with langfuse and opik tracers which do. This means user_id and session_id will always be None in the root span metadata.

🤖 Prompt for AI Agents

In `@src/backend/base/langflow/services/tracing/braintrust.py` around lines 50 -
68, The Braintrust tracer constructor (__init__ in braintrust.py) accepts
user_id and session_id but _initialize_braintrust_tracer in service.py does not
forward them, so root span metadata always has None; update
_initialize_braintrust_tracer to accept (or extract) the service-layer user_id
and session_id and pass them into the Braintrust tracer constructor when
instantiating the tracer (same pattern used by langfuse/opik tracers) so the
root span metadata includes the correct user_id and session_id.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

user_id and session_id are not forwarded to the Braintrust tracer.

Other tracers that support user/session context (e.g., langfuse at line 188–189, opik at line 212–213, traceloop at line 225–226) pass user_id and session_id from trace_context. The Braintrust tracer's __init__ accepts these parameters and includes them in root span metadata, but they will always be None because they aren't passed here.

Proposed fix

         braintrust_tracer = _get_braintrust_tracer()
         trace_context.tracers["braintrust"] = braintrust_tracer(
             trace_name=trace_context.run_name,
             trace_type="chain",
             project_name=trace_context.project_name,
             trace_id=trace_context.run_id,
+            user_id=trace_context.user_id,
+            session_id=trace_context.session_id,
         )

🤖 Prompt for AI Agents

In `@src/backend/base/langflow/services/tracing/service.py` around lines 229 -
238, The Braintrust tracer is not receiving user_id and session_id from the
TraceContext, so root span metadata will always be None; update
_initialize_braintrust_tracer to pass trace_context.user_id and
trace_context.session_id into the braintrust_tracer call (alongside trace_name,
trace_type, project_name, trace_id) so the tracer's constructor receives those
values (TraceContext and braintrust_tracer are the relevant symbols to update).