Skip to content

Telemetry: llm_node span records tool names only - descriptions/schemas never reach Langfuse/OTel backends #6620

Description

@biztex

Feature Type

Improves observability for tool-heavy agents

Feature Description

The llm_node span records the full chat context (lk.chat_ctx) but only the names of function tools (lk.function_tools, generation.py):

  • trace_types.ATTR_FUNCTION_TOOLS: list(tool_ctx.function_tools.keys())

Tool descriptions and parameter schemas are never attached to any span, so OTel backends (Langfuse, Jaeger, etc.) cannot show what the LLM was actually offered on a given turn. When a conversation goes wrong because of a badly-worded tool description or a schema mismatch, the trace has no record of it — you cannot trace back what the model saw. (Reported by a user in the community Slack running Langfuse via set_tracer_provider: tool calls appear, but never the tools' definitions.)

Proposed change

Add a lk.function_tool_definitions span attribute on the llm_node span containing the serialized tool definitions (name, description, parameters schema — the same shape providers receive, via build_legacy_openai_schema for function tools and raw_schema for raw tools). Keep lk.function_tools (names) unchanged for backward compatibility.

Workarounds / Alternatives

Overriding llm_node and setting a custom span attribute per agent; provider SDK debug logging (OPENAI_LOG=debug) outside of tracing. Both are per-team workarounds for something the framework span should carry.

I have a PR ready to submit for this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions