Skip to content

Improve metadata for controller instrumentation #18369

Description

@jaydeluca

Background

The instrumentation documentation system generates docs/instrumentation-list.yaml by running test suites with -PcollectMetadata=true. Each test suite is annotated with a metadataConfig system property whose value becomes the when: key in the generated YAML, allowing readers to understand what telemetry is emitted under which configuration. For example:

systemProperty("metadataConfig", "otel.semconv-stability.opt-in=database")

produces entries like:

telemetry:
- when: otel.semconv-stability.opt-in=database
  spans: ...

The ideal model is: one test suite tagged default (no metadataConfig), plus one additional suite per non-default configuration. This lets a reader see both what they get out of the box and what each opt-in adds.

The controller-telemetry problem

Controller-span instrumentations require otel.instrumentation.common.experimental.controller-telemetry.enabled=true to emit anything; by default they are a no-op. Because there is nothing to observe without the flag, most of these modules run their tests with the flag unconditionally via jvmArgs, and they set metadataConfig to the controller-telemetry option so all telemetry is attributed to that condition.

This is largely correct for pure controller-span modules (e.g. struts-2.3, tapestry-5.4, spring-ws-2.0), but it creates three categories of problems across the ~36 affected modules.


Problem 1: Missing =true suffix in the when condition (spring-webflux-5.0)

spring-webflux-5.0/javaagent/build.gradle.kts sets:

systemProperty("metadataConfig", "otel.instrumentation.common.experimental.controller-telemetry.enabled")
// note: no =true

All other modules use the full form:

systemProperty("metadataConfig", "otel.instrumentation.common.experimental.controller-telemetry.enabled=true")

This causes the generated YAML to produce a non-standard when key:

# spring-webflux-5.0 (inconsistent)
- when: otel.instrumentation.common.experimental.controller-telemetry.enabled

# all other modules (correct)
- when: otel.instrumentation.common.experimental.controller-telemetry.enabled=true

Fix: Add =true to the metadataConfig value in spring-webflux-5.0/javaagent/build.gradle.kts and re-collect telemetry.


Problem 2: Default telemetry buried under the controller-telemetry condition (spring-webflux-5.0)

spring-webflux-5.0 instruments both the Spring WebFlux WebClient (HTTP client spans and http.client.request.duration metric) and controller spans. The WebClient telemetry is emitted by default regardless of whether controller-telemetry is enabled. However, the current test setup runs all tests with jvmArgs("-Dotel.instrumentation.common.experimental.controller-telemetry.enabled=true") and no default test suite, which causes the generated YAML to look like:

telemetry:
- when: otel.instrumentation.common.experimental.controller-telemetry.enabled   # ← all telemetry is here
  metrics:
  - name: http.client.request.duration   # ← actually emitted by default!
  spans:
  - span_kind: CLIENT                    # ← actually emitted by default!
  - span_kind: INTERNAL                  # ← only emitted with the flag

A reader of this documentation would incorrectly conclude that http.client.request.duration and the CLIENT span require the controller-telemetry flag, when in fact they do not.

The same structure appears in the otel.semconv-stability.opt-in=service.peer combined condition entry.

Fix: Add a separate default test suite (no controller-telemetry jvmArg, no metadataConfig) to capture the WebClient telemetry on its own, then a dedicated controller-telemetry suite for the INTERNAL spans only. This separates the two concerns and gives an accurate picture of the default state.


Problem 3: Controller spans incorrectly attributed as default behavior (spring-webmvc-3.1, spring-webmvc-6.0)

spring-webmvc-3.1 and spring-webmvc-6.0 always set the controller-telemetry flag via jvmArgs on every test:

jvmArgs("-Dotel.instrumentation.common.experimental.controller-telemetry.enabled=true")

but they do not set a corresponding metadataConfig for that flag. The only named test suite is testExperimental, which tags telemetry under otel.instrumentation.spring-webmvc.experimental-span-attributes=true. This means the controller INTERNAL spans — which require the flag — land in the untagged (default) bucket:

telemetry:
- when: default          # ← controller spans, which actually require the flag!
  spans:
  - span_kind: INTERNAL
    attributes:
    - name: code.function
    - name: code.namespace

The description even says "controller spans are disabled by default", yet the telemetry shows them as default. This is a direct contradiction.

Fix: Add a dedicated testControllerTelemetry test suite with metadataConfig set to otel.instrumentation.common.experimental.controller-telemetry.enabled=true, and run the default suite without the controller-telemetry jvmArg so the default bucket accurately reflects out-of-the-box behavior (no controller spans).


Scope

Approximately 36 modules set otel.instrumentation.common.experimental.controller-telemetry.enabled=true as a jvmArg. They break down into three categories that need different treatment:

Category Example modules Required action
Pure controller-span modules (no other default telemetry) struts-*, jaxrs-*, jsf-*, tapestry-5.4, spring-ws-2.0, jaxws-*, finatra-2.9 Verify metadataConfig includes =true; no default suite needed since there is nothing to emit by default
Mixed modules with independent default telemetry spring-webflux-5.0, ratpack-1.4 Add a default suite (no controller-telemetry) to capture non-controller telemetry; keep a controller-telemetry suite for the INTERNAL spans
Modules with controller spans leaking into default spring-webmvc-3.1, spring-webmvc-6.0 Split into a true default suite (no flag) and a named controller-telemetry suite

Work to do

  1. Audit the when values in instrumentation-list.yaml for all controller-telemetry entries — find any that are missing =true beyond spring-webflux-5.0.
  2. Cross-reference each affected module's build.gradle.kts to determine which category above it falls into (pure controller-only vs. mixed vs. false-default).
  3. For mixed modules: identify what telemetry the non-controller instrumentation in the same module emits (HTTP client spans, metrics, etc.) and verify it belongs in the when: default bucket.
  4. Fix test suites module by module

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions