|
| 1 | +--- |
| 2 | +id: langsmith |
| 3 | +title: LangSmith integration |
| 4 | +sidebar_label: LangSmith |
| 5 | +toc_max_heading_level: 2 |
| 6 | +keywords: |
| 7 | + - ai |
| 8 | + - agents |
| 9 | + - langsmith |
| 10 | + - observability |
| 11 | + - tracing |
| 12 | +tags: |
| 13 | + - LangSmith |
| 14 | + - Python SDK |
| 15 | + - Temporal SDKs |
| 16 | +description: |
| 17 | + Add LangSmith tracing to Python Workflows using the Temporal Python SDK. |
| 18 | +--- |
| 19 | + |
| 20 | +Temporal's LangSmith integration lets you trace AI agent Workflows in [LangSmith](https://smith.langchain.com/) |
| 21 | +alongside every LLM call, tool execution, and Temporal operation. |
| 22 | + |
| 23 | +Temporal gives your agent code [durable execution](https://docs.temporal.io/temporal#durable-execution). |
| 24 | +LangSmith adds the observability side, so you can inspect LLM inputs and outputs, follow a |
| 25 | +request from the Client through to the model, and compare runs over time. |
| 26 | + |
| 27 | +The `LangSmithPlugin` is what connects the two. It propagates trace context across Temporal boundaries so that runs |
| 28 | +started on the Client nest correctly under Workflow and Activity runs on the Worker. It can also create LangSmith |
| 29 | +runs for Temporal operations themselves: Workflow executions, Activity executions, Signals, Updates, and Queries. |
| 30 | + |
| 31 | +:::tip SUPPORT, STABILITY, and DEPENDENCY INFO |
| 32 | + |
| 33 | +Temporal Python SDK support for LangSmith is at |
| 34 | +[Pre-release](/evaluate/development-production-features/release-stages#pre-release). |
| 35 | + |
| 36 | +All APIs are experimental and may be subject to backwards-incompatible changes. |
| 37 | + |
| 38 | +::: |
| 39 | + |
| 40 | +All code snippets in this guide are taken from the |
| 41 | +[LangSmith tracing sample](https://github.com/temporalio/samples-python/tree/main/langsmith_tracing). Refer to the |
| 42 | +sample for complete code. |
| 43 | + |
| 44 | +## Prerequisites |
| 45 | + |
| 46 | +- This guide assumes you are already familiar with LangSmith. If you aren't, refer to the |
| 47 | + [LangSmith documentation](https://docs.smith.langchain.com/) for more details. |
| 48 | +- If you are new to Temporal, we recommend reading [Understanding Temporal](/evaluate/understanding-temporal) or taking |
| 49 | + the [Temporal 101](https://learn.temporal.io/courses/temporal_101/) course. |
| 50 | +- Ensure you have set up your local development environment by following the |
| 51 | + [Set up your local development environment](/develop/python/set-up-your-local-python) guide. When you're done, leave |
| 52 | + the Temporal Development Server running if you want to test your code locally. |
| 53 | + |
| 54 | +## Configure Workers to use LangSmith |
| 55 | + |
| 56 | +Workers execute the code that defines your Workflows and Activities. To trace Workflow and Activity execution in |
| 57 | +LangSmith, add the `LangSmithPlugin` to your Worker. |
| 58 | + |
| 59 | +Follow the steps below to configure your Worker. |
| 60 | + |
| 61 | +1. Install the Temporal Python SDK with the LangSmith extra. |
| 62 | + |
| 63 | + ```bash |
| 64 | + uv add "temporalio[langsmith]>=1.26.0" |
| 65 | + ``` |
| 66 | + |
| 67 | +2. Add the `LangSmithPlugin` to your Worker. Set `project_name` to the LangSmith project where you want traces to |
| 68 | + appear. |
| 69 | + |
| 70 | + ```python |
| 71 | + from temporalio.contrib.langsmith import LangSmithPlugin |
| 72 | + from temporalio.worker import Worker |
| 73 | + |
| 74 | + worker = Worker( |
| 75 | + client, |
| 76 | + task_queue="my-task-queue", |
| 77 | + workflows=[MyWorkflow], |
| 78 | + activities=[my_activity], |
| 79 | + plugins=[LangSmithPlugin(project_name="my-project")], |
| 80 | + ) |
| 81 | + ``` |
| 82 | + |
| 83 | +3. Run the Worker. Ensure the Worker process has access to your LangSmith API key via the `LANGSMITH_API_KEY` |
| 84 | + environment variable, and enable tracing with `LANGCHAIN_TRACING_V2`. |
| 85 | + |
| 86 | + ```bash |
| 87 | + export LANGSMITH_API_KEY="your-api-key" |
| 88 | + export LANGCHAIN_TRACING_V2=true |
| 89 | + python worker.py |
| 90 | + ``` |
| 91 | + |
| 92 | +## Configure Clients to use LangSmith |
| 93 | + |
| 94 | +Add the plugin to any Temporal Client you use on the Client side (typically a starter or API that calls into |
| 95 | +your Workflows) so that client-side operations like starting a Workflow or sending an Update get linked to the |
| 96 | +Workflows they trigger. |
| 97 | + |
| 98 | +```python |
| 99 | +from temporalio.client import Client |
| 100 | +from temporalio.contrib.langsmith import LangSmithPlugin |
| 101 | + |
| 102 | +client = await Client.connect( |
| 103 | + "localhost:7233", |
| 104 | + plugins=[LangSmithPlugin(project_name="my-project")], |
| 105 | +) |
| 106 | +``` |
| 107 | + |
| 108 | +:::tip |
| 109 | + |
| 110 | +Use the same `project_name` on both the Worker and the Client so their traces land in the same LangSmith project. |
| 111 | + |
| 112 | +::: |
| 113 | + |
| 114 | +:::note |
| 115 | + |
| 116 | +`@traceable` functions on the Client side run outside the plugin's interceptor scope, so they don't pick up |
| 117 | +`project_name` from the plugin. If you have a client-side `@traceable` that wraps a call into your Workflow, pass |
| 118 | +`project_name` to it explicitly so it lands in the same LangSmith project as the rest of the trace. |
| 119 | + |
| 120 | +::: |
| 121 | + |
| 122 | +## Trace Activities |
| 123 | + |
| 124 | +Any non-deterministic work in a Temporal Workflow (LLM calls, tool executions, database queries, external API calls, |
| 125 | +and so on) must run inside an Activity. That makes Activities an important place to add LangSmith runs. When you |
| 126 | +decorate an Activity function with `@traceable`, the run shows up in LangSmith nested under the Workflow that |
| 127 | +scheduled it. |
| 128 | + |
| 129 | +```python |
| 130 | +from dataclasses import dataclass |
| 131 | +from langsmith import traceable |
| 132 | +from temporalio import activity |
| 133 | + |
| 134 | + |
| 135 | +@traceable(name="Fetch Weather", run_type="tool") |
| 136 | +@activity.defn |
| 137 | +async def fetch_weather(city: str) -> str: |
| 138 | + # Call an external weather API here. |
| 139 | + ... |
| 140 | +``` |
| 141 | + |
| 142 | +You can combine `@traceable` with provider-specific LangSmith wrappers to capture more detail. For OpenAI, for |
| 143 | +example, `wrap_openai` patches the client so that every API call creates its own child run with the model name, |
| 144 | +prompt, completion, token counts, and latency. You can access this by wrapping the client: |
| 145 | + |
| 146 | +```python |
| 147 | +from langsmith import traceable |
| 148 | +from langsmith.wrappers import wrap_openai |
| 149 | +from openai import AsyncOpenAI |
| 150 | +from temporalio import activity |
| 151 | + |
| 152 | + |
| 153 | +@dataclass |
| 154 | +class OpenAIRequest: |
| 155 | + model: str |
| 156 | + input: str |
| 157 | + |
| 158 | + |
| 159 | +# wrap_openai patches the client so that every API call adds a ChatOpenAI run under the @traceable. |
| 160 | +# Set max_retries=0 and use Temporal's Activity retry policy instead. |
| 161 | +@traceable(name="Call OpenAI", run_type="llm") |
| 162 | +@activity.defn |
| 163 | +async def call_openai(request: OpenAIRequest) -> str: |
| 164 | + client = wrap_openai(AsyncOpenAI(max_retries=0)) |
| 165 | + response = await client.responses.create( |
| 166 | + model=request.model, |
| 167 | + input=request.input, |
| 168 | + ) |
| 169 | + return response.output_text |
| 170 | +``` |
| 171 | + |
| 172 | +LangSmith ships similar wrappers for |
| 173 | +[Anthropic](https://docs.smith.langchain.com/observability/how-to/integrations#anthropic) and other providers; refer |
| 174 | +to the LangSmith documentation for the full list. |
| 175 | + |
| 176 | +## Add custom runs with @traceable |
| 177 | + |
| 178 | +Decorate functions with `@traceable` to create named runs for your business logic. You control the run name, tags, |
| 179 | +metadata, and `run_type` (`chain`, `llm`, `tool`, `retriever`). |
| 180 | + |
| 181 | +Put `@traceable` on Activities and on private helper methods within your Workflow class that get called from Workflow |
| 182 | +code. For example: |
| 183 | + |
| 184 | +```python |
| 185 | +from langsmith import traceable |
| 186 | +from temporalio import workflow |
| 187 | + |
| 188 | + |
| 189 | +@workflow.defn |
| 190 | +class ChatbotWorkflow: |
| 191 | + # Private helper methods can be decorated directly. |
| 192 | + @traceable(name="Save Note", run_type="tool") |
| 193 | + def _save_note(self, name: str, content: str) -> str: |
| 194 | + ... |
| 195 | +``` |
| 196 | + |
| 197 | +:::warning |
| 198 | + |
| 199 | +Do not put `@traceable` directly on any `@workflow` method (for example, `@workflow.run`, `@workflow.signal`, |
| 200 | +`@workflow.update`, `@workflow.query`). Doing so can produce duplicate or orphaned (unknown parent) runs in LangSmith. |
| 201 | +If you want to trace the body of one of these methods, move the logic into an inner function and decorate that: |
| 202 | + |
| 203 | +```python |
| 204 | +@workflow.defn |
| 205 | +class MyWorkflow: |
| 206 | + @workflow.run |
| 207 | + async def run(self, prompt: str) -> str: |
| 208 | + # Option 1: Use the @traceable decorator |
| 209 | + @traceable(name=f"Ask: {prompt[:60]}", run_type="chain") |
| 210 | + async def _run() -> str: |
| 211 | + ... |
| 212 | + return await _run() |
| 213 | + |
| 214 | + @workflow.update |
| 215 | + async def message_from_user(self, message: str) -> str: |
| 216 | + async def _handle_message(self, message: str) -> str: |
| 217 | + ... |
| 218 | + # Option 2: Use the traceable() function |
| 219 | + return await traceable( |
| 220 | + name=f"Update: {message[:60]}", |
| 221 | + run_type="chain", |
| 222 | + )(self._handle_message)(message) |
| 223 | +``` |
| 224 | + |
| 225 | +::: |
| 226 | + |
| 227 | +## Include Temporal operations as runs |
| 228 | + |
| 229 | +By default, `LangSmithPlugin(add_temporal_runs=False)` only propagates LangSmith context so that `@traceable` and |
| 230 | +`wrap_openai` calls nest correctly. The plugin does not create its own runs. |
| 231 | + |
| 232 | +Set `add_temporal_runs=True` if you want runs for the Temporal operations themselves: Workflow executions, Activity |
| 233 | +executions, Signals, Updates, Queries, and Child Workflows. |
| 234 | + |
| 235 | +```python |
| 236 | +plugin = LangSmithPlugin( |
| 237 | + project_name="my-project", |
| 238 | + add_temporal_runs=True, |
| 239 | +) |
| 240 | +``` |
| 241 | + |
| 242 | +With this on, your LangSmith traces include runs like `StartWorkflow:MyWorkflow`, `RunWorkflow:MyWorkflow`, |
| 243 | +`StartActivity:call_openai`, and `RunActivity:call_openai`. `Start*` and `Run*` pairs appear as siblings: the `Start*` |
| 244 | +run is emitted by the side scheduling the operation (for example, the Client), and the `Run*` run is emitted by the |
| 245 | +side executing it (for example, the Worker). |
| 246 | + |
| 247 | +## Trace hierarchy example |
| 248 | + |
| 249 | +With the plugin configured on both Client and Worker, and `add_temporal_runs=True`, a trace for a simple LLM call looks |
| 250 | +like this: |
| 251 | + |
| 252 | +``` |
| 253 | +Run Agent (@traceable, client-side) |
| 254 | +├── StartWorkflow:MyWorkflow (automatic, LangSmithPlugin) |
| 255 | +└── RunWorkflow:MyWorkflow (automatic, LangSmithPlugin) |
| 256 | + └── Ask: What is Temporal? (@traceable, Workflow) |
| 257 | + ├── StartActivity:call_openai (automatic, LangSmithPlugin) |
| 258 | + └── RunActivity:call_openai (automatic, LangSmithPlugin) |
| 259 | + └── Call OpenAI (@traceable, Activity) |
| 260 | + └── ChatOpenAI (automatic via wrap_openai) |
| 261 | +``` |
| 262 | + |
| 263 | +Without `add_temporal_runs` (the default), only the `@traceable` and `wrap_openai` runs appear. Context still |
| 264 | +propagates, so they nest correctly under the client-side run: |
| 265 | + |
| 266 | +``` |
| 267 | +Run Agent (@traceable, client-side) |
| 268 | +└── Ask: What is Temporal? (@traceable, Workflow-side) |
| 269 | + └── Call OpenAI (@traceable, Activity-side) |
| 270 | + └── ChatOpenAI (automatic via wrap_openai) |
| 271 | +``` |
| 272 | + |
| 273 | +## Example sample |
| 274 | + |
| 275 | +The [LangSmith tracing sample](https://github.com/temporalio/samples-python/tree/main/langsmith_tracing) puts |
| 276 | +these patterns together in two working examples: |
| 277 | + |
| 278 | +- **`basic/`**: a one-shot Workflow that sends a prompt to OpenAI and returns the response. |
| 279 | +- **`chatbot/`**: a long-running conversational Workflow with tool calls (save and read notes), Update handlers, and |
| 280 | + dynamic trace names per message. |
| 281 | + |
| 282 | +Each example shows the `LangSmithPlugin` configuration, `@traceable` runs on the Client, Workflow, and Activity, and |
| 283 | +expected trace output for both `add_temporal_runs=False` and `add_temporal_runs=True`. |
0 commit comments