Add catalog for common configs and metrics in metadata

instrumentation-docs/readme.md

@@ -17,7 +17,7 @@ will enable metric and span collection:
 ```kotlin
 tasks {
   test {
-    systemProperty("collectMetadata", findProperty("collectMetadata"))
+    systemProperty("collectMetadata", otelProps.collectMetadata)
     ...
   }
 }
@@ -32,20 +32,18 @@ For example, to collect and write metadata for the `otel.semconv-stability.opt-i
 set for an instrumentation:
 
 ```kotlin
-val collectMetadata = findProperty("collectMetadata")?.toString() ?: "false"
-
 tasks {
+  withType<Test>().configureEach {
+    systemProperty("collectMetadata", otelProps.collectMetadata)
+  }
+
   val testStableSemconv by registering(Test::class) {
     jvmArgs("-Dotel.semconv-stability.opt-in=database")
 
-    systemProperty("collectMetadata", collectMetadata)
+    systemProperty("collectMetadata", otelProps.collectMetadata)
     systemProperty("metadataConfig", "otel.semconv-stability.opt-in=database")
   }
 
-  test {
-    systemProperty("collectMetadata", collectMetadata)
-  }
-
   check {
     dependsOn(testStableSemconv)
   }
@@ -167,17 +165,65 @@ public class SpringWebInstrumentationModule extends InstrumentationModule
     conforms to. (See [telemetry schema docs](https://opentelemetry.io/docs/specs/otel/schemas/#schema-url))
   - attributes: The instrumentation scope’s optional attributes provide additional information
     about the scope.
-- configuration settings
-  - List of settings that are available for the instrumentation module
-  - Each setting has a name (flat property format), optional declarative_name (YAML path format), description, type, default value, and optional examples
-  - `name` is optional for declarative-only settings that have no flat property (they must provide a `declarative_name`)
-  - Structured-list settings additionally carry `declarative_type: structured_list` and a `declarative_schema` describing the per-item object shape
-- metrics
-  - List of metrics that the instrumentation module collects, including the metric name, description, type, and attributes.
-  - Separate lists for the metrics emitted by default vs via configuration options.
+- configuration_refs
+  - List of configuration definition ids the instrumentation module exposes.
+  - Each id resolves to an entry in the top-level `definitions.configurations` catalog (see below).
+  - This is the _generated_ representation. In a module's `metadata.yaml` you still author each
+    configuration either inline or as a shared `ref` (see "metadata.yaml file" and "Shared
+    configuration definitions" below); the generator collects both forms into the catalog and emits
+    the ids here.
+- metrics (referenced via `metric_refs`)
+  - Each telemetry `when` block lists `metric_refs`: the ids of metric definitions the module emits.
+  - Each id resolves to an entry in the top-level `definitions.metrics` catalog.
+  - Separate `when` blocks distinguish metrics emitted by default vs via configuration options.
+  - This is the _generated_ representation, produced automatically from collected telemetry (and any
+    inline `additional_telemetry`). Metrics are always authored inline (or auto-collected from
+    tests) — there is no author-time shared metric catalog; the generator does the de-duplication
+    into `definitions.metrics` for you.
 - spans
-  - List of spans kinds the instrumentation module generates, including the attributes and their types.
-  - Separate lists for the spans emitted by default vs via configuration options.
+  - List of span kinds the instrumentation module generates, including the attributes and their types.
+  - Emitted inline under each telemetry `when` block (spans are not yet part of the definitions catalog).
+  - Separate `when` blocks distinguish spans emitted by default vs via configuration options.
+
+### Definitions catalog
+
+To avoid inlining the same metric or configuration block into every instrumentation entry, common
+definitions are collected once into a top-level `definitions` section and referenced by id:
+
+```yaml
+definitions:
+  configurations:
+    http.known-methods:          # id, reused as the configuration_refs value
+      name: otel.instrumentation.http.known-methods
+      declarative_name: java.common.http.known_methods
+      description: ...
+      type: list
+      default: CONNECT,DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT,TRACE
+  metrics:
+    http.client.request.duration-1a2b3c4d:   # <metric name>-<short content hash>
+      name: http.client.request.duration
+      description: Duration of HTTP client requests.
+      instrument: histogram
+      data_type: HISTOGRAM
+      unit: s
+      attributes: [...]
+libraries:
+- name: java-http-client
+  configuration_refs:
+  - http.known-methods
+  telemetry:
+  - when: default
+    metric_refs:
+    - http.client.request.duration-1a2b3c4d
+```
+
+Configuration ids are the curated ids from the shared registry (see below) for shared options, or the
+config `name` for module-specific options. If two module-specific options share the same `name` but
+have different content, their ids are disambiguated as `<config name>-<short content hash>`. Metric
+ids are `<metric name>-<short content hash>`; the
+hash disambiguates metrics that share a name but have different attribute sets. Any consumer of
+`instrumentation-list.yaml` must resolve `configuration_refs` / `metric_refs` against `definitions`
+before reading the underlying config/metric content.
 
 ## Methodology
 
@@ -201,9 +247,14 @@ disabled_by_default: true                         # Defaults to `false`
 classification: internal                          # instrumentation classification: library | internal | custom
 library_link: https://...                         # URL to the library or framework's main website or documentation
 configurations:
-  - name: otel.instrumentation.common.db.query-sanitization.enabled
-    declarative_name: java.common.db.query_sanitization.enabled    # Optional: YAML config path
-    description: Enables query sanitization for database queries.
+  # Reference a shared definition instead of hand-copying a common config block. The id must exist
+  # in instrumentation-docs/src/main/resources/shared-config-definitions.yaml (an unknown ref fails
+  # the build). See "Shared configuration definitions" below.
+  - ref: common.db.query-sanitization.enabled
+  # Module-specific configs are still defined inline:
+  - name: otel.instrumentation.my-module.enabled
+    declarative_name: java.my_module.enabled    # Optional: YAML config path
+    description: Enables the my-module feature.
     type: boolean               # boolean | string | list | map (the flat form)
     default: true
     examples:                   # Optional: Example values for this configuration
@@ -248,6 +299,28 @@ additional_telemetry:                             # Manually document telemetry
             type: "STRING"
 ```
 
+### Shared configuration definitions
+
+Many instrumentations expose the same configuration options (for example `http.known-methods` or
+`common.peer-service-mapping`). Instead of hand-copying an identical block into every `metadata.yaml`,
+these are defined once in
+[`src/main/resources/shared-config-definitions.yaml`](src/main/resources/shared-config-definitions.yaml)
+under a stable id, and each module references them:
+
+```yaml
+configurations:
+  - ref: http.known-methods
+  - ref: common.peer-service-mapping
+```
+
+At metadata-parse time (`SharedConfigurationRegistry`) each `ref` is expanded into the full
+configuration option; an unknown id fails the build. The same ids are reused as the keys in the
+generated `definitions.configurations` catalog, so editing a description in the registry once updates
+every module that references it.
+
+To add a new shared option, add an entry to `shared-config-definitions.yaml` and reference it by id.
+To change an option for a single module only, define it inline instead of using a `ref`.
+
 ### Gradle File Derived Information
 
 We parse gradle files in order to determine several pieces of metadata:

instrumentation-docs/src/main/java/io/opentelemetry/instrumentation/docs/DocGeneratorApplication.java

@@ -43,9 +43,13 @@ public static void main(String[] args) throws IOException {
         Files.newBufferedWriter(Paths.get(baseRepoPath + "docs/instrumentation-list.yaml"))) {
       writer.write("# This file is generated and should not be manually edited.\n");
       writer.write("# The structure and contents are a work in progress and subject to change.\n");
+      writer.write(
+          "# Common metrics and configurations are collected in the top-level `definitions` catalog;\n");
+      writer.write(
+          "# each module references them by id via `metric_refs` and `configuration_refs`.\n");
       writer.write(
           "# For more information see: https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/13468\n\n");
-      writer.write("file_format: 0.5\n\n");
+      writer.write("file_format: 0.6\n\n");
       YamlHelper.generateInstrumentationYaml(modules, writer);
     }
 

instrumentation-docs/src/main/java/io/opentelemetry/instrumentation/docs/internal/ConfigurationOption.java

@@ -7,6 +7,7 @@
 
 import static java.util.Objects.requireNonNull;
 
+import com.fasterxml.jackson.annotation.JsonIgnore;
 import com.fasterxml.jackson.annotation.JsonProperty;
 import java.util.List;
 import javax.annotation.Nullable;
@@ -23,40 +24,86 @@ public record ConfigurationOption(
     ConfigurationType type,
     @Nullable List<String> examples,
     @JsonProperty("declarative_type") @Nullable ConfigurationType declarativeType,
-    @JsonProperty("declarative_schema") @Nullable DeclarativeSchema declarativeSchema) {
+    @JsonProperty("declarative_schema") @Nullable DeclarativeSchema declarativeSchema,
+    @Nullable String ref,
+    // The definition id is assigned internally via withId() during registry resolution; it must
+    // never be supplied from metadata.yaml, otherwise an inline option could claim a registry id
+    // and
+    // overwrite the shared definition in buildDefinitionCatalog.
+    @JsonIgnore @Nullable String id) {
 
   public ConfigurationOption {
-    requireNonNull(description, "description");
-    requireNonNull(defaultValue, "defaultValue");
-    requireNonNull(type, "type");
-
-    // Most configs are backed by a flat system property (name). Declarative-only configs (such as
-    // the url_template_rules structured list) have no flat property and rely on declarative_name.
-    if (name == null && declarativeName == null) {
-      throw new IllegalArgumentException(
-          "ConfigurationOption must have a name or a declarative_name");
-    }
-    if ((name != null && name.isBlank()) || description.isBlank()) {
-      throw new IllegalArgumentException("ConfigurationOption name/description cannot be blank");
-    }
-
-    if (declarativeType == ConfigurationType.STRUCTURED_LIST) {
-      if (declarativeSchema == null
-          || declarativeSchema.properties() == null
-          || declarativeSchema.properties().isEmpty()) {
+    // A reference entry (`- ref: <id>`) in a metadata.yaml carries only the ref id; it is resolved
+    // against the shared configuration registry before use, so it skips the field validation that
+    // applies to fully-specified options. Because resolve() replaces the entire entry with the
+    // shared definition, any other field set alongside the ref would be silently discarded. Reject
+    // such entries (and a blank ref) so authors cannot publish settings that appear effective but
+    // are ignored.
+    if (ref != null) {
+      if (ref.isBlank()) {
+        throw new IllegalArgumentException("A ref ConfigurationOption must not have a blank ref");
+      }
+      if (name != null
+          || declarativeName != null
+          || description != null
+          || defaultValue != null
+          || type != null
+          || examples != null
+          || declarativeType != null
+          || declarativeSchema != null
+          || id != null) {
         throw new IllegalArgumentException(
-            "structured_list configuration must define a declarative_schema with properties");
+            "A ref ConfigurationOption must not specify any other fields; it carries only the ref"
+                + " id");
       }
-      if (declarativeSchema.required() != null
-          && !declarativeSchema.properties().keySet().containsAll(declarativeSchema.required())) {
+    } else {
+      requireNonNull(description, "description");
+      requireNonNull(defaultValue, "defaultValue");
+      requireNonNull(type, "type");
+
+      // Most configs are backed by a flat system property (name). Declarative-only configs (such as
+      // the url_template_rules structured list) have no flat property and rely on declarative_name.
+      if (name == null && declarativeName == null) {
         throw new IllegalArgumentException(
-            "declarative_schema required keys must be a subset of its properties");
+            "ConfigurationOption must have a name or a declarative_name");
+      }
+      if ((name != null && name.isBlank()) || description.isBlank()) {
+        throw new IllegalArgumentException("ConfigurationOption name/description cannot be blank");
+      }
+
+      if (declarativeType == ConfigurationType.STRUCTURED_LIST) {
+        if (declarativeSchema == null
+            || declarativeSchema.properties() == null
+            || declarativeSchema.properties().isEmpty()) {
+          throw new IllegalArgumentException(
+              "structured_list configuration must define a declarative_schema with properties");
+        }
+        if (declarativeSchema.required() != null
+            && !declarativeSchema.properties().keySet().containsAll(declarativeSchema.required())) {
+          throw new IllegalArgumentException(
+              "declarative_schema required keys must be a subset of its properties");
+        }
       }
     }
   }
 
   public ConfigurationOption(
       String name, String description, String defaultValue, ConfigurationType type) {
-    this(name, null, description, defaultValue, type, null, null, null);
+    this(name, null, description, defaultValue, type, null, null, null, null, null);
+  }
+
+  /** Returns a copy of this option with the given definition id assigned. */
+  public ConfigurationOption withId(String id) {
+    return new ConfigurationOption(
+        name,
+        declarativeName,
+        description,
+        defaultValue,
+        type,
+        examples,
+        declarativeType,
+        declarativeSchema,
+        ref,
+        id);
   }
 }

instrumentation-docs/src/main/java/io/opentelemetry/instrumentation/docs/internal/SharedConfigurationRegistry.java

@@ -0,0 +1,92 @@
+/*
+ * Copyright The OpenTelemetry Authors
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package io.opentelemetry.instrumentation.docs.internal;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.UncheckedIOException;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Loads the shared configuration definitions ("globals") from {@code
+ * shared-config-definitions.yaml} and resolves {@code ref} entries in a module's configuration list
+ * into full {@link ConfigurationOption}s.
+ *
+ * <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
+ * at any time.
+ */
+public final class SharedConfigurationRegistry {
+
+  private static final String RESOURCE = "/shared-config-definitions.yaml";
+
+  private static final SharedConfigurationRegistry INSTANCE =
+      new SharedConfigurationRegistry(load());
+
+  private final Map<String, ConfigurationOption> definitions;
+
+  SharedConfigurationRegistry(Map<String, ConfigurationOption> definitions) {
+    this.definitions = definitions;
+  }
+
+  public static SharedConfigurationRegistry getInstance() {
+    return INSTANCE;
+  }
+
+  /** Returns the shared definitions keyed by id. */
+  public Map<String, ConfigurationOption> definitions() {
+    return definitions;
+  }
+
+  /**
+   * Expands each {@code ref} entry into its shared definition (tagged with the definition id) and
+   * passes non-ref entries through unchanged. Throws {@link IllegalArgumentException} on an unknown
+   * ref id so a mistyped ref fails the build rather than silently dropping a config.
+   */
+  public List<ConfigurationOption> resolve(List<ConfigurationOption> configurations) {
+    List<ConfigurationOption> resolved = new ArrayList<>(configurations.size());
+    for (ConfigurationOption configuration : configurations) {
+      if (configuration.ref() != null) {
+        ConfigurationOption definition = definitions.get(configuration.ref());
+        if (definition == null) {
+          throw new IllegalArgumentException(
+              "Unknown shared configuration ref: '"
+                  + configuration.ref()
+                  + "'. Known ids: "
+                  + definitions.keySet());
+        }
+        resolved.add(definition);
+      } else {
+        resolved.add(configuration);
+      }
+    }
+    return resolved;
+  }
+
+  private static Map<String, ConfigurationOption> load() {
+    ObjectMapper mapper = new ObjectMapper(new YAMLFactory());
+    try (InputStream stream = SharedConfigurationRegistry.class.getResourceAsStream(RESOURCE)) {
+      if (stream == null) {
+        throw new IllegalStateException("Missing shared config definitions resource: " + RESOURCE);
+      }
+      RegistryFile file = mapper.readValue(stream, RegistryFile.class);
+      Map<String, ConfigurationOption> byId = new LinkedHashMap<>();
+      Map<String, ConfigurationOption> configurations = file.configurations();
+      if (configurations != null) {
+        configurations.forEach((id, option) -> byId.put(id, option.withId(id)));
+      }
+      return byId;
+    } catch (IOException e) {
+      throw new UncheckedIOException("Failed to read " + RESOURCE, e);
+    }
+  }
+
+  private record RegistryFile(Map<String, ConfigurationOption> configurations) {}
+}