Skip to content

feat(instrumentation-cohere): migrate to OTel 1.40 GenAI semantic conventions#927

Open
nuthalapativarun wants to merge 2 commits intotraceloop:mainfrom
nuthalapativarun:feat/cohere-otel-1-40-migration
Open

feat(instrumentation-cohere): migrate to OTel 1.40 GenAI semantic conventions#927
nuthalapativarun wants to merge 2 commits intotraceloop:mainfrom
nuthalapativarun:feat/cohere-otel-1-40-migration

Conversation

@nuthalapativarun
Copy link
Copy Markdown

@nuthalapativarun nuthalapativarun commented Apr 12, 2026

Summary

  • Migrates instrumentation-cohere to align with OTel GenAI semantic conventions 1.40
  • Replaces deprecated SpanAttributes.LLM_* with new ATTR_GEN_AI_* constants from @opentelemetry/semantic-conventions/incubating
  • Uses formatInputMessages / formatOutputMessage from @traceloop/instrumentation-utils for JSON message formatting
  • Adds cohereRoleMap to normalize Cohere roles (CHATBOTassistant) to OTel-standard values
  • Adds cohereFinishReasonMap to normalize Cohere finish reasons to OTel standard values
  • Updates all test files to assert against new attribute names

Closes #58

🤖 Generated with Claude Code

…ventions

Replace deprecated LLM span attributes with OTel 1.40 GenAI semconv:
- ATTR_GEN_AI_SYSTEM -> ATTR_GEN_AI_PROVIDER_NAME (GEN_AI_PROVIDER_NAME_VALUE_COHERE)
- SpanAttributes.LLM_REQUEST_TYPE -> ATTR_GEN_AI_OPERATION_NAME
- SpanAttributes.LLM_TOP_K -> ATTR_GEN_AI_REQUEST_TOP_K
- ATTR_GEN_AI_USAGE_PROMPT_TOKENS -> ATTR_GEN_AI_USAGE_INPUT_TOKENS
- ATTR_GEN_AI_USAGE_COMPLETION_TOKENS -> ATTR_GEN_AI_USAGE_OUTPUT_TOKENS
- gen_ai.prompt/completion indexed attributes -> ATTR_GEN_AI_INPUT/OUTPUT_MESSAGES (JSON)
- Add ATTR_GEN_AI_RESPONSE_FINISH_REASONS with normalized finish reason mapping
- Use formatInputMessages/formatInputMessagesFromPrompt/formatOutputMessage
  from @traceloop/instrumentation-utils
- Span name format: "${operation} ${model}" (e.g., "chat command")

Co-Authored-By: Varun Nuthalapati <nuthalapativarun@gmail.com>
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 12, 2026

CLA assistant check
All committers have signed the CLA.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 12, 2026

📝 Walkthrough

Walkthrough

The PR updates the Cohere instrumentation package to align with OpenTelemetry's incubating GEN-AI semantic conventions. Changes include adding a utility dependency, refactoring attribute mappings from legacy conventions to standardized GEN-AI attributes, introducing typed operation names, and updating all test assertions to verify the new attributes.

Changes

Cohort / File(s) Summary
Dependency Management
packages/instrumentation-cohere/package.json
Added @traceloop/instrumentation-utils as a workspace dependency.
Core Instrumentation Refactor
packages/instrumentation-cohere/src/instrumentation.ts
Replaced legacy LLM/completion attributes with OTel GEN-AI conventions: ATTR_GEN_AI_PROVIDER_NAME, ATTR_GEN_AI_OPERATION_NAME, ATTR_GEN_AI_REQUEST_MODEL, input/output message attributes, and standardized token usage attributes. Introduced typed operation-name constants and cohereFinishReasonMap for mapping finish reasons. Refactored message capture via format helpers and span naming from cohere.${type} to ${type} ${model}.
Test Updates
packages/instrumentation-cohere/tests/chat.test.ts, packages/instrumentation-cohere/tests/generate.test.ts, packages/instrumentation-cohere/tests/rerank.test.ts
Updated span attribute assertions to verify new GEN-AI semantic conventions: removed legacy ATTR_GEN_AI_SYSTEM/LLM_REQUEST_TYPE checks, added provider/operation name assertions, replaced prompt/completion token attributes with input/output token attributes, and verified presence of ATTR_GEN_AI_INPUT_MESSAGES and ATTR_GEN_AI_OUTPUT_MESSAGES.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 With whiskers twitched and ears held high,
We trace through Cohere's API sky—
New attributes dance, old ones fade,
Semantic conventions perfectly made!
OpenTelemetry's path we now follow,
Making observability less hollow! 🌟

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title directly and accurately summarizes the main change: migration of the Cohere instrumentation package to OTel 1.40 GenAI semantic conventions.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/instrumentation-cohere/src/instrumentation.ts`:
- Around line 77-78: The mapping key named COMPLETE_lower doesn't match the
provider's lowercase "complete" value, so update the mapping by replacing the
symbol COMPLETE_lower with the actual string key "complete" (preserve the mapped
value FinishReasons.STOP) so lowercase generate responses are normalized to the
OTel finish reason; locate the mapping object that references COMPLETE_lower and
change the key to "complete" while keeping the value as FinishReasons.STOP.
- Around line 420-424: The code incorrectly maps
result.meta?.billedUnits?.searchUnits to
SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS; remove that mapping so searchUnits is
not recorded as total tokens. Update the instrumentation in the function
handling Rerank responses (the block that checks
result.meta?.billedUnits?.searchUnits and calls span.setAttribute with
SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS) by deleting that setAttribute call, or
if you need to capture searchUnits separately, record it under a new attribute
(e.g., SpanAttributes.GEN_AI_USAGE_RERANK_SEARCH_UNITS) and add that enum/key to
SpanAttributes instead of using GEN_AI_USAGE_TOTAL_TOKENS.
- Around line 311-320: The messages array building for
ATTR_GEN_AI_INPUT_MESSAGES currently lowercases Cohere roles (in the map over
params.chatHistory) which turns CHATBOT into "chatbot" instead of the OTel
standard "assistant"; update the mapping inside that map to normalize roles
(e.g., const role = (msg.role ?? "").toString().toUpperCase(); switch/map
USER->"user", CHATBOT->"assistant", SYSTEM->"system",
default->msg.role.toLowerCase()) and use that normalized role when constructing
each message passed to formatInputMessages; keep using formatInputMessages and
ATTR_GEN_AI_INPUT_MESSAGES as-is.
- Around line 509-514: The code incorrectly sets ATTR_GEN_AI_RESPONSE_MODEL from
result.responseId; remove the block that checks "responseId" in result and calls
span.setAttribute(ATTR_GEN_AI_RESPONSE_MODEL, result.responseId). Instead rely
on the existing ATTR_GEN_AI_REQUEST_MODEL (populated from params.model) for
model info and do not populate ATTR_GEN_AI_RESPONSE_MODEL with responseId
anywhere in the instrumentation (remove references to result.responseId ->
ATTR_GEN_AI_RESPONSE_MODEL, leaving other span attributes unchanged).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ae680c86-1446-4085-ab40-bba870e97ebe

📥 Commits

Reviewing files that changed from the base of the PR and between a869545 and 7d942c5.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (5)
  • packages/instrumentation-cohere/package.json
  • packages/instrumentation-cohere/src/instrumentation.ts
  • packages/instrumentation-cohere/tests/chat.test.ts
  • packages/instrumentation-cohere/tests/generate.test.ts
  • packages/instrumentation-cohere/tests/rerank.test.ts

- Fix COMPLETE_lower -> "complete" key in finish reason map so lowercase
  generate API responses are properly normalized to OTel FinishReasons.STOP
- Normalize Cohere chat history roles: CHATBOT -> "assistant", USER -> "user",
  SYSTEM -> "system" per OTel GenAI semantic conventions
- Remove searchUnits mapped to GEN_AI_USAGE_TOTAL_TOKENS; searchUnits is a
  billing unit specific to Rerank, not a token count
- Remove responseId set as ATTR_GEN_AI_RESPONSE_MODEL; responseId is an
  opaque response ID, not a model identifier

Co-Authored-By: Varun Nuthalapati <nuthalapativarun@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Re-enable VertexAI tests

2 participants