Skip to content

Commit e1c8167

Browse files
keith-deckertammy-baylis-swilmolkova
authored
GenAI Documentation Update (#4369)
* update readme * Update util/opentelemetry-util-genai/README.rst Co-authored-by: Tammy Baylis <96076570+tammy-baylis-swi@users.noreply.github.com> * Update util/opentelemetry-util-genai/README.rst Co-authored-by: Liudmila Molkova <neskazu@gmail.com> --------- Co-authored-by: Tammy Baylis <96076570+tammy-baylis-swi@users.noreply.github.com> Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
1 parent 828138a commit e1c8167

File tree

1 file changed

+123
-48
lines changed

1 file changed

+123
-48
lines changed

‎util/opentelemetry-util-genai/README.rst‎

Lines changed: 123 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,58 +1,129 @@
11
OpenTelemetry Util for GenAI
22
============================
33

4+
The GenAI Utils package provides boilerplate and helpers to standardize instrumentation for Generative AI.
5+
It offers APIs to minimize the work needed to instrument GenAI libraries,
6+
while providing standardization for generating spans, metrics, and events.
47

5-
The GenAI Utils package will include boilerplate and helpers to standardize instrumentation for Generative AI.
6-
This package will provide APIs and decorators to minimize the work needed to instrument genai libraries,
7-
while providing standardization for generating both types of otel, "spans and metrics" and "spans, metrics and events"
88

9-
This package relies on environment variables to configure capturing of message content.
9+
Key Components
10+
--------------
11+
12+
- ``TelemetryHandler`` -- manages LLM invocation lifecycles (spans, metrics, events)
13+
- ``LLMInvocation`` and message types (``Text``, ``Reasoning``, ``Blob``, etc.) -- structured data model for GenAI interactions
14+
- ``CompletionHook`` -- protocol for uploading content to external storage (built-in ``fsspec`` support)
15+
- Metrics -- ``gen_ai.client.operation.duration`` and ``gen_ai.client.token.usage`` histograms
16+
17+
18+
Usage
19+
-----
20+
21+
See the module docstring in ``opentelemetry.util.genai.handler`` for usage examples,
22+
including context manager and manual lifecycle patterns.
23+
24+
25+
Environment Variables
26+
---------------------
27+
28+
This package relies on environment variables to configure capturing of message content.
1029
By default, message content will not be captured.
11-
Set the environment variable `OTEL_SEMCONV_STABILITY_OPT_IN` to `gen_ai_latest_experimental` to enable experimental features.
12-
Set the environment variable `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` to one of:
13-
- `NO_CONTENT`: Do not capture message content (default).
14-
- `SPAN_ONLY`: Capture message content in spans only.
15-
- `EVENT_ONLY`: Capture message content in events only.
16-
- `SPAN_AND_EVENT`: Capture message content in both spans and events.
17-
18-
To control event emission, you can optionally set `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` to `true` or `false` (case-insensitive).
19-
This variable controls whether to emit `gen_ai.client.inference.operation.details` events.
20-
If not explicitly set, the default value is automatically determined by `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`:
21-
- When `NO_CONTENT` or `SPAN_ONLY` is set: defaults to `false`
22-
- When `EVENT_ONLY` or `SPAN_AND_EVENT` is set: defaults to `true`
30+
Set the environment variable ``OTEL_SEMCONV_STABILITY_OPT_IN`` to ``gen_ai_latest_experimental`` to enable experimental features.
31+
Set the environment variable ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`` to one of:
32+
33+
- ``NO_CONTENT``: Do not capture message content (default).
34+
- ``SPAN_ONLY``: Capture message content in spans only.
35+
- ``EVENT_ONLY``: Capture message content in events only.
36+
- ``SPAN_AND_EVENT``: Capture message content in both spans and events.
37+
38+
To control event emission, you can optionally set ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` to ``true`` or ``false`` (case-insensitive).
39+
This variable controls whether to emit ``gen_ai.client.inference.operation.details`` events.
40+
If not explicitly set, the default value is automatically determined by ``OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT``:
41+
42+
- When ``NO_CONTENT`` or ``SPAN_ONLY`` is set: defaults to ``false``
43+
- When ``EVENT_ONLY`` or ``SPAN_AND_EVENT`` is set: defaults to ``true``
44+
2345
If explicitly set, the user's value takes precedence over the default.
2446

25-
This package provides these span attributes:
26-
27-
- `gen_ai.provider.name`: Str(openai)
28-
- `gen_ai.operation.name`: Str(chat)
29-
- `gen_ai.request.model`: Str(gpt-3.5-turbo)
30-
- `gen_ai.response.finish_reasons`: Slice(["stop"])
31-
- `gen_ai.response.model`: Str(gpt-3.5-turbo-0125)
32-
- `gen_ai.response.id`: Str(chatcmpl-Bz8yrvPnydD9pObv625n2CGBPHS13)
33-
- `gen_ai.usage.input_tokens`: Int(24)
34-
- `gen_ai.usage.output_tokens`: Int(7)
35-
- `gen_ai.input.messages`: Str('[{"role": "Human", "parts": [{"content": "hello world", "type": "text"}]}]')
36-
- `gen_ai.output.messages`: Str('[{"role": "AI", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]')
37-
- `gen_ai.system_instructions`: Str('[{"content": "You are a helpful assistant.", "type": "text"}]') (when system instruction is provided)
38-
39-
This package also supports embedding invocation spans via the `embedding` context manager.
40-
For embedding invocations, common attributes include:
41-
42-
- `gen_ai.provider.name`: Str(openai)
43-
- `gen_ai.operation.name`: Str(embeddings)
44-
- `gen_ai.request.model`: Str(text-embedding-3-small)
45-
- `gen_ai.embeddings.dimension.count`: Int(1536)
46-
- `gen_ai.request.encoding_formats`: Slice(["float"])
47-
- `gen_ai.usage.input_tokens`: Int(24)
48-
- `server.address`: Str(api.openai.com)
49-
- `server.port`: Int(443)
50-
51-
When `EVENT_ONLY` or `SPAN_AND_EVENT` mode is enabled and a LoggerProvider is configured,
52-
the package also emits `gen_ai.client.inference.operation.details` events with structured
53-
message content (as dictionaries instead of JSON strings). Note that when using `EVENT_ONLY`
54-
or `SPAN_AND_EVENT`, the `OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT` environment variable defaults
55-
to `true`, so events will be emitted automatically unless explicitly set to `false`.
47+
When ``EVENT_ONLY`` or ``SPAN_AND_EVENT`` mode is enabled and a LoggerProvider is configured,
48+
the package also emits ``gen_ai.client.inference.operation.details`` events with structured
49+
message content (as dictionaries instead of JSON strings). Note that when using ``EVENT_ONLY``
50+
or ``SPAN_AND_EVENT``, the ``OTEL_INSTRUMENTATION_GENAI_EMIT_EVENT`` environment variable defaults
51+
to ``true``, so events will be emitted automatically unless explicitly set to ``false``.
52+
53+
Completion Hook / Upload
54+
^^^^^^^^^^^^^^^^^^^^^^^^
55+
56+
- ``OTEL_INSTRUMENTATION_GENAI_COMPLETION_HOOK``: Name of the completion hook entry point to load (e.g. ``upload``).
57+
- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_BASE_PATH``: An ``fsspec``-compatible URI/path for uploading prompts and responses
58+
(e.g. ``/path/to/prompts`` or ``gs://my_bucket``). Required when using the ``upload`` hook.
59+
- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_FORMAT``: Format for uploaded data -- ``json`` (default) or ``jsonl``.
60+
- ``OTEL_INSTRUMENTATION_GENAI_UPLOAD_MAX_QUEUE_SIZE``: Maximum number of concurrent uploads to queue (default: ``20``).
61+
62+
63+
Span Attributes
64+
---------------
65+
66+
This package sets the following span attributes on LLM invocations:
67+
68+
**Common attributes:**
69+
70+
- ``gen_ai.operation.name``: Str(chat)
71+
- ``gen_ai.provider.name``: Str(openai)
72+
- ``gen_ai.request.model``: Str(gpt-4o)
73+
- ``server.address``: Str(api.openai.com)
74+
- ``server.port``: Int(443)
75+
76+
**Response attributes:**
77+
78+
- ``gen_ai.response.finish_reasons``: Slice(["stop"])
79+
- ``gen_ai.response.model``: Str(gpt-4o-2024-05-13)
80+
- ``gen_ai.response.id``: Str(chatcmpl-Bz8yrvPnydD9pObv625n2CGBPHS13)
81+
- ``gen_ai.usage.input_tokens``: Int(24)
82+
- ``gen_ai.usage.output_tokens``: Int(7)
83+
84+
**Request parameter attributes (when provided):**
85+
86+
- ``gen_ai.request.temperature``: Float(0.7)
87+
- ``gen_ai.request.top_p``: Float(1.0)
88+
- ``gen_ai.request.frequency_penalty``: Float(0.0)
89+
- ``gen_ai.request.presence_penalty``: Float(0.0)
90+
- ``gen_ai.request.max_tokens``: Int(1024)
91+
- ``gen_ai.request.stop_sequences``: Slice(["\\n"])
92+
- ``gen_ai.request.seed``: Int(42)
93+
94+
**Content attributes (sensitive, requires content capturing enabled):**
95+
96+
- ``gen_ai.input.messages``: Str('[{"role": "user", "parts": [{"content": "hello world", "type": "text"}]}]')
97+
- ``gen_ai.output.messages``: Str('[{"role": "assistant", "parts": [{"content": "hello back", "type": "text"}], "finish_reason": "stop"}]')
98+
- ``gen_ai.system_instructions``: Str('[{"content": "You are a helpful assistant.", "type": "text"}]')
99+
100+
**Error attributes:**
101+
102+
- ``error.type``: Str(TimeoutError)
103+
104+
Embedding Span Attributes
105+
-------------------------
106+
107+
This package also supports embedding invocation spans via the ``embedding`` context manager.
108+
For embedding invocations, the following attributes are set:
109+
110+
**Common attributes:**
111+
112+
- ``gen_ai.operation.name``: Str(embeddings)
113+
- ``gen_ai.provider.name``: Str(openai)
114+
- ``server.address``: Str(api.openai.com)
115+
- ``server.port``: Int(443)
116+
117+
**Request attributes:**
118+
119+
- ``gen_ai.request.model``: Str(text-embedding-3-small)
120+
- ``gen_ai.embeddings.dimension.count``: Int(1536)
121+
- ``gen_ai.request.encoding_formats``: Slice(["float"])
122+
123+
**Response attributes:**
124+
125+
- ``gen_ai.response.model``: Str(text-embedding-3-small)
126+
- ``gen_ai.usage.input_tokens``: Int(24)
56127

57128

58129
Installation
@@ -62,11 +133,15 @@ Installation
62133

63134
pip install opentelemetry-util-genai
64135

136+
For upload support (requires ``fsspec``)::
137+
138+
pip install opentelemetry-util-genai[upload]
139+
65140

66141
Design Document
67142
---------------
68143

69-
The design document for the OpenTelemetry GenAI Utils can be found at: `Design Document <https://docs.google.com/document/d/1w9TbtKjuRX_wymS8DRSwPA03_VhrGlyx65hHAdNik1E/edit?tab=t.qneb4vabc1wc#heading=h.kh4j6stirken>`_
144+
The design document for the OpenTelemetry GenAI Utils can be found at: `Design Document <https://docs.google.com/document/d/1LzNGylxot5zaIV1goOJZ2mz3LI0weFu1SgaMnyDv7gg/edit?usp=sharing>`_
70145

71146
References
72147
----------

0 commit comments

Comments
 (0)