feat(genai-openai): OpenAI responses extractors

[eternalcuriouslearner]

Description

This change adds strongly typed Responses API extractors for openai-v2 and reduces dependence on Any, based on feedback we received on the sync Anthropic SDK instrumentation. Pydantic models are used
here to:

• normalize dict and object inputs
• avoid Any
• validate only the fields we care about
• fail safely by logging and returning empty values instead of breaking telemetry

Dependencies required for this change:

• pydantic >= 2, < 3

Fixes #3436 partly

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Focused extractor tests were added/updated to cover:

• request validation failures returning empty values and logging
• response validation failures returning empty values and logging
• response usage extraction from object and mapping inputs
• capture_content=True populating output_messages
• prevalidated response models skipping revalidation
• Responses API status-to-finish-reason mapping behavior

Reproduce with:

python3 -m pytest --noconftest instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_response_extractors.py

Test result in local environment:

• Focused test_response_extractors.py passes (14 tests)

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[xrmx]

Question: isn't openai using pydantic itself and so aren't its answer expected to be of the expected shape?

[eternalcuriouslearner]

Question: isn't openai using pydantic itself and so aren't its answer expected to be of the expected shape?

Howdy Riccardo, yep OpenAI uses pydantic. We’re using Pydantic here to validate and normalize the minimal request/response shapes that the instrumentation depends on. The local models let us avoid Any, handle the supported inputs through one path, and fail closed by logging and returning empty values instead of letting extraction errors break telemetry. In a nutshell I am using pydantic for following reasons:

• normalize dict and object inputs
• avoid Any
• validate only the fields we care about
• fail safely by logging and returning empty values instead of breaking telemetry

Lmk your thoughts.

CC: @xrmx @aabmass

[MikeGoldsmith]

Looks good to me. I've left a suggestion around testing but not blocking.