Skip to content

readme.md

Latest commit

 

History

History
410 lines (343 loc) · 16 KB

readme.md

File metadata and controls

410 lines (343 loc) · 16 KB
Raw

Doc Generator

Runs analysis on instrumentation modules in order to generate documentation.

How to use

Run the analysis to update the instrumentation-list.yaml:

./gradlew :instrumentation-docs:runAnalysis

Telemetry collection

Until this process is ready for all instrumentations, each module will be modified to include a system property feature flag configured for when the tests run. By enabling the following flag you will enable metric and span collection:

tasks {
  test {
    systemProperty("collectMetadata", otelProps.collectMetadata)
    ...
  }
}

Sometimes instrumentation will behave differently based on configuration options, and we can differentiate between these configurations by using the metadataConfig system property. When the telemetry is written to a file, the value of this property will be included, or it will default to a default attribution.

For example, to collect and write metadata for the otel.semconv-stability.opt-in=database option set for an instrumentation:

tasks {
  withType<Test>().configureEach {
    systemProperty("collectMetadata", otelProps.collectMetadata)
  }

  val testStableSemconv by registering(Test::class) {
    jvmArgs("-Dotel.semconv-stability.opt-in=database")

    systemProperty("collectMetadata", otelProps.collectMetadata)
    systemProperty("metadataConfig", "otel.semconv-stability.opt-in=database")
  }

  check {
    dependsOn(testStableSemconv)
  }
}

Then, prior to running the analyzer, run the following command to generate .telemetry files:

./gradlew test -PcollectMetadata=true

Then run the doc generator

./gradlew :instrumentation-docs:runAnalysis

or use the helper script that will run only the currently supported tests (recommended):

./instrumentation-docs/collect.sh

Instrumentation Hierarchy

An "InstrumentationModule" represents a module that targets specific code in a framework/library/technology. Each module will have a name, a namespace, and a group.

Using these structures as examples:

├── instrumentation
│   ├── clickhouse-client-05
│   ├── jaxrs
│   │   ├── jaxrs-1.0
│   │   ├── jaxrs-2.0
│   ├── spring
│   │   ├── spring-cloud-gateway
│   │   │   ├── spring-cloud-gateway-2.0
│   │   │   ├── spring-cloud-gateway-2.2
│   │   │   └── spring-cloud-gateway-common

Results in the following:

  • Name - the full name of the instrumentation module
    • clickhouse-client-05, jaxrs-1.0, spring-cloud-gateway-2.0
  • Namespace - direct parent. if none, use name and strip version
    • clickhouse-client, jaxrs, spring-cloud-gateway
  • Group - top most parent
    • clickhouse-client, jaxrs, spring

This information is also referenced in InstrumentationModule code for each module:

public class SpringWebInstrumentationModule extends InstrumentationModule
    implements ExperimentalInstrumentationModule {
  public SpringWebInstrumentationModule() {
    super("spring-web", "spring-web-3.1");
  }

Instrumentation metadata

  • classification
    • library - Instrumentation that targets a library
    • internal - Instrumentation that is used internally by the OpenTelemetry Java Agent
    • custom - Utilities that are used to create custom instrumentation
  • name
    • Identifier for instrumentation module, used to enable/disable
    • Configured in InstrumentationModule code for each module
  • semantic_conventions
    • The semantic conventions that the instrumentation module adheres to
    • Options are:
      • HTTP_CLIENT_SPANS
      • HTTP_CLIENT_METRICS
      • HTTP_SERVER_SPANS
      • HTTP_SERVER_METRICS
      • RPC_CLIENT_SPANS
      • RPC_CLIENT_METRICS
      • RPC_SERVER_SPANS
      • RPC_SERVER_METRICS
      • MESSAGING_SPANS
      • DATABASE_CLIENT_SPANS
      • DATABASE_CLIENT_METRICS
      • DATABASE_POOL_METRICS
      • JVM_RUNTIME_METRICS
      • GRAPHQL_SERVER_SPANS
      • FAAS_SERVER_SPANS
      • GENAI_CLIENT_SPANS
      • GENAI_CLIENT_METRIC
  • features
    • The specific functionality that the instrumentation provides
    • Options are:
      • HTTP_ROUTE
      • CONTEXT_PROPAGATION
      • AUTO_INSTRUMENTATION_SHIM
      • CONTROLLER_SPANS
      • VIEW_SPANS
      • LOGGING_BRIDGE
      • RESOURCE_DETECTOR
  • library_link
    • URL to the library or framework's main website or documentation, or if those don't exist, the GitHub repository.
  • source_path
    • Path to the source code of the instrumentation module
  • minimum_java_version
    • Minimum Java version required by the instrumentation module. If not specified, it is assumed to be Java 8
  • description
    • Short description of what the instrumentation does
  • has_standalone_library
    • Whether the instrumentation module has a standalone library that can be used outside of the agent
  • has_javaagent
    • Whether the instrumentation module has a javaagent component that can be used with the OpenTelemetry Java Agent
  • target_versions
    • List of supported versions by the module, broken down by library or javaagent support
  • scope (See instrumentation-scope docs)
    • name: The scope name of the instrumentation, io.opentelemetry.{instrumentation name}
    • schema_url: Location of the telemetry schema that the instrumentation’s emitted telemetry conforms to. (See telemetry schema docs)
    • attributes: The instrumentation scope’s optional attributes provide additional information about the scope.
  • configuration_refs
    • List of configuration definition ids the instrumentation module exposes.
    • Each id resolves to an entry in the top-level definitions.configurations catalog (see below).
    • This is the generated representation. In a module's metadata.yaml you still author each configuration either inline or as a shared ref (see "metadata.yaml file" and "Shared configuration definitions" below); the generator collects both forms into the catalog and emits the ids here.
  • metrics (referenced via metric_refs)
    • Each telemetry when block lists metric_refs: the ids of metric definitions the module emits.
    • Each id resolves to an entry in the top-level definitions.metrics catalog.
    • Separate when blocks distinguish metrics emitted by default vs via configuration options.
    • This is the generated representation, produced automatically from collected telemetry (and any inline additional_telemetry). Metrics are always authored inline (or auto-collected from tests) — there is no author-time shared metric catalog; the generator does the de-duplication into definitions.metrics for you.
  • spans
    • List of span kinds the instrumentation module generates, including the attributes and their types.
    • Emitted inline under each telemetry when block (spans are not yet part of the definitions catalog).
    • Separate when blocks distinguish spans emitted by default vs via configuration options.

Definitions catalog

To avoid inlining the same metric or configuration block into every instrumentation entry, common definitions are collected once into a top-level definitions section and referenced by id:

definitions:
  configurations:
    http.known-methods:          # id, reused as the configuration_refs value
      name: otel.instrumentation.http.known-methods
      declarative_name: java.common.http.known_methods
      description: ...
      type: list
      default: CONNECT,DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT,TRACE
  metrics:
    http.client.request.duration-1a2b3c4d:   # <metric name>-<short content hash>
      name: http.client.request.duration
      description: Duration of HTTP client requests.
      instrument: histogram
      data_type: HISTOGRAM
      unit: s
      attributes: [...]
libraries:
- name: java-http-client
  configuration_refs:
  - http.known-methods
  telemetry:
  - when: default
    metric_refs:
    - http.client.request.duration-1a2b3c4d

Configuration ids are the curated ids from the shared registry (see below) for shared options, or the config name for module-specific options. If two module-specific options share the same name but have different content, their ids are disambiguated as <config name>-<short content hash>. Metric ids are <metric name>-<short content hash>; the hash disambiguates metrics that share a name but have different attribute sets. Any consumer of instrumentation-list.yaml must resolve configuration_refs / metric_refs against definitions before reading the underlying config/metric content.

Methodology

metadata.yaml file

Within each instrumentation source directory, a metadata.yaml file can be created to provide additional information about the instrumentation module.

As of now, the following fields are supported, all of which are optional:

description: "This instrumentation enables..."    # Description of the instrumentation module
semantic_conventions:                             # List of semantic conventions the instrumentation adheres to
  - HTTP_CLIENT_SPANS
  - DATABASE_CLIENT_SPANS
  - JVM_RUNTIME_METRICS
features:                                        # List of features this instrumentation provides
  - HTTP_ROUTE
  - CONTEXT_PROPAGATION
disabled_by_default: true                         # Defaults to `false`
classification: internal                          # instrumentation classification: library | internal | custom
library_link: https://...                         # URL to the library or framework's main website or documentation
configurations:
  # Reference a shared definition instead of hand-copying a common config block. The id must exist
  # in instrumentation-docs/src/main/resources/shared-config-definitions.yaml (an unknown ref fails
  # the build). See "Shared configuration definitions" below.
  - ref: common.db.query-sanitization.enabled
  # Module-specific configs are still defined inline:
  - name: otel.instrumentation.my-module.enabled
    declarative_name: java.my_module.enabled    # Optional: YAML config path
    description: Enables the my-module feature.
    type: boolean               # boolean | string | list | map (the flat form)
    default: true
    examples:                   # Optional: Example values for this configuration
      - "true"
      - "false"
  # Structured-list config: a list of objects in declarative config. `name` may be omitted for
  # declarative-only settings (identified solely by `declarative_name`).
  - declarative_name: java.common.http.client.url_template_rules
    description: Rules for deriving low-cardinality URL templates from HTTP client request URLs.
    type: list
    default: ""
    declarative_type: structured_list      # Optional: overrides the declarative-form shape
    declarative_schema:                     # Required when declarative_type is structured_list
      type: object
      required: [pattern, template]
      properties:
        pattern:
          type: string
          description: Regular expression matched against the request URL.
        template:
          type: string
          description: Template used to derive the low-cardinality route.
        override:
          type: boolean
          default: false
          description: Whether this rule overrides an already-applied template.
override_telemetry: false                         # Set to true to ignore auto-generated .telemetry files
additional_telemetry:                             # Manually document telemetry metadata
  - when: "default"
    metrics:
      - name: "metric.name"
        description: "Metric description"
        type: "COUNTER"
        unit: "1"
        attributes:
          - name: "attribute.name"
            type: "STRING"
    spans:
      - span_kind: "CLIENT"
        attributes:
          - name: "span.attribute"
            type: "STRING"

Shared configuration definitions

Many instrumentations expose the same configuration options (for example http.known-methods or common.peer-service-mapping). Instead of hand-copying an identical block into every metadata.yaml, these are defined once in src/main/resources/shared-config-definitions.yaml under a stable id, and each module references them:

configurations:
  - ref: http.known-methods
  - ref: common.peer-service-mapping

At metadata-parse time (SharedConfigurationRegistry) each ref is expanded into the full configuration option; an unknown id fails the build. The same ids are reused as the keys in the generated definitions.configurations catalog, so editing a description in the registry once updates every module that references it.

To add a new shared option, add an entry to shared-config-definitions.yaml and reference it by id. To change an option for a single module only, define it inline instead of using a ref.

Gradle File Derived Information

We parse gradle files in order to determine several pieces of metadata:

  • Javaagent versions are determined by the muzzle plugin configurations
  • Standalone Library versions are identified, but we do not try and parse version ranges
  • Minimum Java version is determined by the otelJava configurations

Scope

For now, the scope name is the only value that is implemented in our instrumentations. The scope name is determined by the instrumentation module name: io.opentelemetry.{instrumentation name}

We will implement gatherers for the schemaUrl and scope attributes when instrumentations start implementing them.

Spans and Metrics

In order to identify what telemetry is emitted from instrumentations, we can hook into the InstrumentationTestRunner class and collect the metrics and spans generated during runs. We can then leverage the afterTestClass() in the Agent and library test runners to then write this information into temporary files. When we analyze the instrumentation modules, we can read these files and generate the telemetry section of the instrumentation-list.yaml file.

The data is written into a .telemetry directory in the root of each instrumentation module. This data will be excluded from git and just generated on demand.

Each file has a when value along with the list of metrics that indicates whether the telemetry is emitted by default or via a configuration option.

Manual Telemetry Documentation

In addition to auto-generated telemetry data from test runs, you can manually document telemetry metadata directly in the metadata.yaml file. This is useful for:

  • Documenting telemetry that may not be captured during test runs
  • Overriding auto-generated telemetry data when it's incomplete or incorrect
  • Adding additional telemetry documentation that complements the auto-generated data

You can add manual telemetry documentation using the additional_telemetry field:

additional_telemetry:
  - when: "default"  # or any configuration condition
    metrics:
      - name: "my.custom.metric"
        description: "Description of the metric"
        type: "COUNTER"
        unit: "1"
        attributes:
          - name: "attribute.name"
            type: "STRING"
    spans:
      - span_kind: "CLIENT"
        attributes:
          - name: "span.attribute"
            type: "STRING"

To completely replace auto-generated telemetry data (ignoring .telemetry files), set override_telemetry: true:

override_telemetry: true
additional_telemetry:
  - when: "default"
    metrics:
      - name: "documented.metric"
        description: "This replaces all auto-generated metrics"
        type: "GAUGE"
        unit: "ms"

When both manual and auto-generated telemetry exist for the same when condition, they are merged with manual entries taking precedence in case of conflicts (same metric name or span kind).

Doc Synchronization

The documentation site has a section that lists all the instrumentations in the context of documenting how to disable them.

We have a class DocSynchronization that runs a check against our instrumentation-list.yaml file to identify when we have missing entries, so we know to go update them.

You can run this via:

./gradlew :instrumentation-docs:docSiteAudit

This is setup to run nightly in a github action.