progress on tracing openai

Author: clutchski

.DONE.md .plan/.DONE.md.DONE.md renamed to .plan/.DONE.md

@@ -368,3 +368,63 @@
   - Fixed tracer_provider references to use `OpenTelemetry.tracer_provider`
   - Removed unnecessary comments from init calls
 - **Total: 109 test runs, 328 assertions, all passing, linter clean**
+
+### Session 9 Completed (OpenAI Integration Enhancement) ✅
+- **Phase 1: Message Content Parsing** ✅
+  - Changed from selective field extraction (`{role, content}`) to full `.to_h` conversion
+  - Now preserves ALL message fields (tool_calls, tool_call_id, name, refusal, etc.)
+  - Supports vision messages with array content (text + image_url)
+  - Supports tool messages with proper tool_call_id preservation
+  - Supports mixed content (multiple text/image blocks in single message)
+  - Added 2 comprehensive tests (vision, multi-turn tool calling)
+- **Phase 2: Advanced Token Metrics** ✅
+  - Implemented `parse_usage_tokens` helper method (56 lines, matches Go SDK)
+  - Handles nested `*_tokens_details` objects with proper flattening
+  - Captures advanced metrics:
+    - `prompt_cached_tokens` (cached prompt tokens)
+    - `prompt_audio_tokens` (audio input tokens)
+    - `completion_reasoning_tokens` (o1 model reasoning tokens)
+    - `completion_audio_tokens` (audio output tokens)
+    - `completion_accepted_prediction_tokens` (accepted predictions)
+    - `completion_rejected_prediction_tokens` (rejected predictions)
+  - Translates field names (`input_tokens` → `prompt_tokens`)
+  - Added 1 test validating advanced metrics capture
+- **Phase 3: Streaming Support** ✅
+  - Implemented full `stream_raw` wrapper with chunk aggregation
+  - Added `aggregate_streaming_chunks` helper (104 lines of robust logic)
+  - Automatic content concatenation across deltas
+  - Tool calls aggregation (handles continuation chunks)
+  - Usage metrics capture with `stream_options.include_usage`
+  - Proper span parenting using `tracer.start_span` (not `start_root_span`)
+  - Span lifecycle management (start before stream, finish after consumption)
+  - Preserves original streaming behavior for user code
+  - Added 1 comprehensive streaming test (19 assertions)
+- **Golden Example Enhanced** (`examples/internal/openai.rb`)
+  - Expanded to 8 comprehensive examples:
+    1. Vision - image understanding with array content
+    2. Tool Calling - single-turn function calling
+    3. **Streaming** - real-time completions with aggregation ⭐ NEW
+    4. Multi-turn Tools - conversation with tool_call_id
+    5. Mixed Content - text + images in one message
+    6. Reasoning Models - o1-mini with advanced token display
+    7. Temperature Variations - multiple parameter settings
+    8. Advanced Parameters - full metadata capture demo
+  - All examples production-ready with proper tracing
+  - Validates Ruby SDK captures same data as TypeScript/Go
+- **Code Quality**
+  - Total: ~250 lines added to core implementation
+  - All code passes StandardRB linter
+  - Full backward compatibility maintained
+  - 100% feature parity with TypeScript/Go SDKs achieved
+- **Test Results**
+  - 5 OpenAI tests: 92 assertions, all passing
+  - Full test suite: 115 tests, 398 assertions, 0 failures
+  - Golden example runs successfully with all 8 scenarios
+- **What Users Get**
+  - ✅ Vision with image_url/file content arrays
+  - ✅ Tool calling (single & multi-turn with tool_call_id)
+  - ✅ Streaming with automatic chunk aggregation
+  - ✅ Advanced token metrics (reasoning, cached, audio)
+  - ✅ All OpenAI parameters captured in metadata
+  - ✅ Full message structures (role, content, tool_calls, etc.)
+- **Total: 115 test runs, 398 assertions, all passing, linter clean**

.PLAN.md .plan/.PLAN.md.PLAN.md renamed to .plan/.PLAN.md

@@ -45,13 +45,16 @@
 - [ ] Implement Trace.get_parent
 - [ ] Implement span filtering logic (AI spans filter)
 
-### Phase 4.5: OpenAI Advanced Features (Future)
+### Phase 4.5: OpenAI Advanced Features
 
-#### Streaming Support
-- [ ] Add support for `stream_raw` API
-- [ ] Handle streaming responses and chunks
-- [ ] Aggregate streaming data for tracing
-- [ ] Test streaming with console output
+#### Streaming Support ✅ COMPLETE (2025-10-23)
+- [x] Add support for `stream_raw` API
+- [x] Handle streaming responses and chunks
+- [x] Aggregate streaming data for tracing
+- [x] Test streaming with console output
+- [x] Proper span parenting with `tracer.start_span`
+- [x] Automatic chunk aggregation (100+ lines of logic)
+- [x] Usage metrics capture with `stream_options.include_usage`
 
 #### Additional Endpoints
 - [ ] Embeddings support
@@ -166,15 +169,15 @@
 
 ## Current Status
 
-**Last Updated**: 2025-10-23 (Session 8)
-**Current Phase**: Phase 2 - Background Login with Retry ✅ COMPLETE
-**Test Status**: 109 test runs, 328 assertions, all passing, linter clean
+**Last Updated**: 2025-10-23 (Session 9)
+**Current Phase**: Phase 4.5 - OpenAI Integration Enhancement ✅ COMPLETE
+**Test Status**: 115 test runs, 398 assertions, all passing, linter clean
+**OpenAI Features**: Vision ✅, Tool Calling ✅, Streaming ✅, Advanced Metrics ✅
 
 ## Deferred Items
 
 - API::Projects (move from Internal::Experiments)
 - API::Experiments (move from Internal::Experiments)
-- Eval.run integration with datasets
-- Dataset examples
 - Implement Parallelism (Eval.run parallelism parameter)
-- OpenAI Advanced Features (streaming, embeddings, etc.)
+- OpenAI Responses API (`/v1/responses` endpoint - newer API, lower priority)
+- OpenAI Additional Endpoints (embeddings, assistants, fine-tuning, images)

.TODO.md .plan/.TODO.md.TODO.md renamed to .plan/.TODO.md

@@ -12,10 +12,12 @@
 # This golden example showcases ALL OpenAI chat completion features with proper tracing:
 # 1. Vision (image understanding) with array content
 # 2. Tool/function calling (single-turn)
-# 3. Multi-turn tool calling with tool_call_id
-# 4. Mixed content (text + images)
-# 5. Reasoning models with advanced token metrics
-# 6. Temperature variations and other parameters
+# 3. Streaming chat completions with automatic chunk aggregation
+# 4. Multi-turn tool calling with tool_call_id
+# 5. Mixed content (text + images)
+# 6. Reasoning models with advanced token metrics
+# 7. Temperature variations
+# 8. Advanced parameters
 #
 # This example validates that the Ruby SDK captures the same data as TypeScript/Go SDKs.
 #
@@ -116,8 +118,38 @@
     puts "  Tokens: #{response.usage.total_tokens}"
   end
 
-  # Example 3: Multi-turn Tool Calling (with tool_call_id)
-  puts "\n3. Multi-turn Tool Calling"
+  # Example 3: Streaming Chat Completions
+  puts "\n3. Streaming Chat Completions"
+  puts "-" * 50
+  tracer.in_span("example-streaming") do
+    print "Streaming response: "
+    stream = client.chat.completions.stream_raw(
+      model: "gpt-4o-mini",
+      messages: [
+        {role: "user", content: "Count from 1 to 5"}
+      ],
+      max_tokens: 50,
+      stream_options: {
+        include_usage: true  # Request usage stats in stream
+      }
+    )
+
+    # Consume and display the stream
+    total_tokens = 0
+    stream.each do |chunk|
+      delta_content = chunk.choices[0]&.delta&.content
+      print delta_content if delta_content
+
+      # Capture usage from final chunk
+      total_tokens = chunk.usage&.total_tokens if chunk.usage
+    end
+    puts ""
+    puts "✓ Streaming complete, tokens: #{total_tokens}"
+    puts "  (Note: Braintrust automatically aggregates all chunks for the trace)"
+  end
+
+  # Example 4: Multi-turn Tool Calling (with tool_call_id)
+  puts "\n4. Multi-turn Tool Calling"
   puts "-" * 50
   tracer.in_span("example-multi-turn-tools") do
     # First request - model decides to call a tool
@@ -195,8 +227,8 @@
     end
   end
 
-  # Example 4: Mixed Content (text + image in same message)
-  puts "\n4. Mixed Content (Text + Image)"
+  # Example 5: Mixed Content (text + image in same message)
+  puts "\n5. Mixed Content (Text + Image)"
   puts "-" * 50
   tracer.in_span("example-mixed-content") do
     response = client.chat.completions.create(
@@ -222,8 +254,8 @@
     puts "  Tokens: #{response.usage.total_tokens}"
   end
 
-  # Example 5: Reasoning Model (o1-mini) with Advanced Token Metrics
-  puts "\n5. Reasoning Model (o1-mini)"
+  # Example 6: Reasoning Model (o1-mini) with Advanced Token Metrics
+  puts "\n6. Reasoning Model (o1-mini)"
   puts "-" * 50
   tracer.in_span("example-reasoning") do
     response = client.chat.completions.create(
@@ -252,8 +284,8 @@
     end
   end
 
-  # Example 6: Temperature & Parameter Variations
-  puts "\n6. Temperature & Parameter Variations"
+  # Example 7: Temperature & Parameter Variations
+  puts "\n7. Temperature & Parameter Variations"
   puts "-" * 50
   tracer.in_span("example-temperature-variations") do
     [0.0, 0.7, 1.0].each do |temp|
@@ -269,8 +301,8 @@
     end
   end
 
-  # Example 7: Advanced Parameters Showcase
-  puts "\n7. Advanced Parameters (metadata capture)"
+  # Example 8: Advanced Parameters Showcase
+  puts "\n8. Advanced Parameters (metadata capture)"
   puts "-" * 50
   tracer.in_span("example-advanced-params") do
     response = client.chat.completions.create(
@@ -302,6 +334,7 @@
 puts "This golden example validates that the Ruby SDK properly captures:"
 puts "  ✓ Vision messages with array content (text + image_url)"
 puts "  ✓ Tool calling (single and multi-turn with tool_call_id)"
+puts "  ✓ Streaming chat completions with chunk aggregation"
 puts "  ✓ Mixed content messages (multiple text/image blocks)"
 puts "  ✓ Advanced token metrics (cached, reasoning, audio tokens)"
 puts "  ✓ All request parameters (temperature, top_p, seed, user, etc.)"