feat(instrumentation-cohere): migrate to OTel 1.40 GenAI semantic conventions#927
feat(instrumentation-cohere): migrate to OTel 1.40 GenAI semantic conventions#927nuthalapativarun wants to merge 2 commits intotraceloop:mainfrom
Conversation
…ventions
Replace deprecated LLM span attributes with OTel 1.40 GenAI semconv:
- ATTR_GEN_AI_SYSTEM -> ATTR_GEN_AI_PROVIDER_NAME (GEN_AI_PROVIDER_NAME_VALUE_COHERE)
- SpanAttributes.LLM_REQUEST_TYPE -> ATTR_GEN_AI_OPERATION_NAME
- SpanAttributes.LLM_TOP_K -> ATTR_GEN_AI_REQUEST_TOP_K
- ATTR_GEN_AI_USAGE_PROMPT_TOKENS -> ATTR_GEN_AI_USAGE_INPUT_TOKENS
- ATTR_GEN_AI_USAGE_COMPLETION_TOKENS -> ATTR_GEN_AI_USAGE_OUTPUT_TOKENS
- gen_ai.prompt/completion indexed attributes -> ATTR_GEN_AI_INPUT/OUTPUT_MESSAGES (JSON)
- Add ATTR_GEN_AI_RESPONSE_FINISH_REASONS with normalized finish reason mapping
- Use formatInputMessages/formatInputMessagesFromPrompt/formatOutputMessage
from @traceloop/instrumentation-utils
- Span name format: "${operation} ${model}" (e.g., "chat command")
Co-Authored-By: Varun Nuthalapati <nuthalapativarun@gmail.com>
📝 WalkthroughWalkthroughThe PR updates the Cohere instrumentation package to align with OpenTelemetry's incubating GEN-AI semantic conventions. Changes include adding a utility dependency, refactoring attribute mappings from legacy conventions to standardized GEN-AI attributes, introducing typed operation names, and updating all test assertions to verify the new attributes. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/instrumentation-cohere/src/instrumentation.ts`:
- Around line 77-78: The mapping key named COMPLETE_lower doesn't match the
provider's lowercase "complete" value, so update the mapping by replacing the
symbol COMPLETE_lower with the actual string key "complete" (preserve the mapped
value FinishReasons.STOP) so lowercase generate responses are normalized to the
OTel finish reason; locate the mapping object that references COMPLETE_lower and
change the key to "complete" while keeping the value as FinishReasons.STOP.
- Around line 420-424: The code incorrectly maps
result.meta?.billedUnits?.searchUnits to
SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS; remove that mapping so searchUnits is
not recorded as total tokens. Update the instrumentation in the function
handling Rerank responses (the block that checks
result.meta?.billedUnits?.searchUnits and calls span.setAttribute with
SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS) by deleting that setAttribute call, or
if you need to capture searchUnits separately, record it under a new attribute
(e.g., SpanAttributes.GEN_AI_USAGE_RERANK_SEARCH_UNITS) and add that enum/key to
SpanAttributes instead of using GEN_AI_USAGE_TOTAL_TOKENS.
- Around line 311-320: The messages array building for
ATTR_GEN_AI_INPUT_MESSAGES currently lowercases Cohere roles (in the map over
params.chatHistory) which turns CHATBOT into "chatbot" instead of the OTel
standard "assistant"; update the mapping inside that map to normalize roles
(e.g., const role = (msg.role ?? "").toString().toUpperCase(); switch/map
USER->"user", CHATBOT->"assistant", SYSTEM->"system",
default->msg.role.toLowerCase()) and use that normalized role when constructing
each message passed to formatInputMessages; keep using formatInputMessages and
ATTR_GEN_AI_INPUT_MESSAGES as-is.
- Around line 509-514: The code incorrectly sets ATTR_GEN_AI_RESPONSE_MODEL from
result.responseId; remove the block that checks "responseId" in result and calls
span.setAttribute(ATTR_GEN_AI_RESPONSE_MODEL, result.responseId). Instead rely
on the existing ATTR_GEN_AI_REQUEST_MODEL (populated from params.model) for
model info and do not populate ATTR_GEN_AI_RESPONSE_MODEL with responseId
anywhere in the instrumentation (remove references to result.responseId ->
ATTR_GEN_AI_RESPONSE_MODEL, leaving other span attributes unchanged).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ae680c86-1446-4085-ab40-bba870e97ebe
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (5)
packages/instrumentation-cohere/package.jsonpackages/instrumentation-cohere/src/instrumentation.tspackages/instrumentation-cohere/tests/chat.test.tspackages/instrumentation-cohere/tests/generate.test.tspackages/instrumentation-cohere/tests/rerank.test.ts
- Fix COMPLETE_lower -> "complete" key in finish reason map so lowercase generate API responses are properly normalized to OTel FinishReasons.STOP - Normalize Cohere chat history roles: CHATBOT -> "assistant", USER -> "user", SYSTEM -> "system" per OTel GenAI semantic conventions - Remove searchUnits mapped to GEN_AI_USAGE_TOTAL_TOKENS; searchUnits is a billing unit specific to Rerank, not a token count - Remove responseId set as ATTR_GEN_AI_RESPONSE_MODEL; responseId is an opaque response ID, not a model identifier Co-Authored-By: Varun Nuthalapati <nuthalapativarun@gmail.com>
Summary
instrumentation-cohereto align with OTel GenAI semantic conventions 1.40SpanAttributes.LLM_*with newATTR_GEN_AI_*constants from@opentelemetry/semantic-conventions/incubatingformatInputMessages/formatOutputMessagefrom@traceloop/instrumentation-utilsfor JSON message formattingcohereRoleMapto normalize Cohere roles (CHATBOT→assistant) to OTel-standard valuescohereFinishReasonMapto normalize Cohere finish reasons to OTel standard valuesCloses #58
🤖 Generated with Claude Code