Skip to content

Propagate simulator device/OS tags from inner test spans to runner spans#283

Merged
ypopovych merged 3 commits into
mainfrom
fix/propagate-simulator-device-tags-to-runner-spans
Jun 16, 2026
Merged

Propagate simulator device/OS tags from inner test spans to runner spans#283
ypopovych merged 3 commits into
mainfrom
fix/propagate-simulator-device-tags-to-runner-spans

Conversation

@ypopovych

@ypopovych ypopovych commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The integration test runner always executes on macOS, so its DDTest spans were tagged with macOS device info even when the inner tests ran on a simulator (iOS, tvOS, watchOS, visionOS).
  • After xcodebuild finishes and spans are collected by MockBackend, copy os.platform, os.architecture, os.version, device.name, and device.model from the first received span onto DDTest.current.
  • The propagation is skipped when the inner platform is "macOS" — in that case the runner's own tags are already correct.

Test plan

  • Run integration tests targeting an iOS simulator and verify the runner test spans in Datadog show simulator device info (e.g. iOS simulator, iPhone 16) instead of macOS host info.
  • Run integration tests targeting macOS and verify no tag overriding occurs.

🤖 Generated with Claude Code

The integration test runner always executes on macOS, so its DDTest spans
were tagged with macOS device info even when the inner tests ran on a
simulator (iOS, tvOS, watchOS, visionOS). After xcodebuild finishes, copy
os.platform, os.architecture, os.version, device.name, and device.model
from the first received span onto DDTest.current — but only when the inner
platform is not macOS, where the runner tags are already correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ypopovych
ypopovych requested review from a team as code owners June 16, 2026 15:25
ypopovych and others added 2 commits June 16, 2026 17:28
…ectly

Device/OS tags are written into the envelope-level metadata under the
"test_levels" key by the SDK. SpanEvent.extend only merges metadata keyed
by span.type ("test", "span", etc.) and "*", so span.meta never contains
os.platform — making the previous guard always fail and the whole block
get skipped.

Fix: read from envelope.metadata["test_levels"] directly. For each device
tag key, prefer a value found in individual span meta (spans can override
via set(tag:)), falling back to the envelope metadata value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ns only

Merge priority (highest to lowest):
  span.meta > metadata[span.type] > metadata["test_levels"] > metadata["*"]

metadata["test_levels"] carries device/OS tags shared across all test-level
events. It is applied only to test* spans (.test, suiteEnd, moduleEnd,
sessionEnd) — not to generic "span" events where it does not belong.

With test_levels now correctly merged into extended spans, the runner's
device tag propagation can read from allSpans.meta directly (reverted from
the previous direct envelope.metadata["test_levels"] workaround).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@datadog-prod-us1-5

Copy link
Copy Markdown

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

Integration Tests | Unit Tests / Xcode_26.2 / tvOSsim   View in Datadog   GitHub Actions

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a433909 | Docs | Datadog PR Page | Give us feedback!

@ypopovych
ypopovych merged commit 65b0cd5 into main Jun 16, 2026
30 of 31 checks passed
@ypopovych
ypopovych deleted the fix/propagate-simulator-device-tags-to-runner-spans branch June 16, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants