GenAI Utils | Add support for ToolCall Invocations#4356
GenAI Utils | Add support for ToolCall Invocations#4356keith-decker wants to merge 51 commits intoopen-telemetry:mainfrom
Conversation
…e GenericPart for custom message types
…pentelemetry-python-contrib into pr1-enhance-toolcall-type
3b05b19 to
a9f9eef
Compare
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py
Outdated
Show resolved
Hide resolved
| """Fail an LLM invocation and end its span with error status.""" | ||
| return self._fail(invocation, error) | ||
|
|
||
| @contextmanager |
There was a problem hiding this comment.
In the google gen AI instrumentation we monkey patch the actual function call: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumentation-genai/opentelemetry-instrumentation-google-genai/src/opentelemetry/instrumentation/google_genai/tool_call_wrapper.py#L164
IDK if it makes sense to do something like that here or not (accept the actual Callable object)?
Either way I'll try to update that instrumentation to use this once we submit
eternalcuriouslearner
left a comment
There was a problem hiding this comment.
Overall LGTM!! I left few questions to understand the design better.
util/opentelemetry-util-genai/src/opentelemetry/util/genai/span_utils.py
Show resolved
Hide resolved
| # Metric-related fields (for gen_ai.client.operation.duration) | ||
| provider: str | None = None # gen_ai.provider.name (Required for metrics) | ||
| server_address: str | None = None # server.address (Recommended) | ||
| server_port: int | None = None # server.port (Conditionally Required) |
There was a problem hiding this comment.
question: Why are we ignoring attributes: dict[str, Any] = field(default_factory=_new_str_any_dict) attribute here?
There was a problem hiding this comment.
These are properties used by the metrics. the attributes you mentioned is used for custom attributes the user might want to use on the span.
There was a problem hiding this comment.
Ahh dumb question are we ignoring custom attributes for ToolCall?
Description
Implementing the ToolCall handler methods. Check out the example app here
Some output from the example app:
Fixes # (issue)
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.