[chore] Add implementation details to Component Configuration Management RFC

.github/workflows/utils/cspell.json

@@ -178,6 +178,7 @@
       "crosslink",
       "cumulativetodeltaprocessor",
       "customname",
+      "custompkg",
       "dataloss",
       "datapoints",
       "debugexporter",
@@ -270,6 +271,8 @@
       "httpprovider",
       "httpsprovider",
       "httptest",
+      "icmpcheck",
+      "icmpcheckreceiver",
       "illumos",
       "incorrectclass",
       "incorrectcomponent",
@@ -316,6 +319,7 @@
       "mowies",
       "muehle",
       "multiclient",
+      "multierr",
       "multimod",
       "mycert",
       "mycomponent",

docs/rfcs/component-configuration-schema-roadmap.md

@@ -66,9 +66,7 @@ Use of the `schemagen` tool is dictated by the modularity of the Collector compo
 
 ```yaml
 config:
-  allOf:
-    - $ref: "go.opentelemetry.io/collector/scraper/scraperhelper#/$defs/ControllerConfig"
-    - $ref: "#/$defs/MetricsBuilderConfig"
+  type: object
   properties:
     targets:
       type: array
@@ -77,27 +75,267 @@ config:
         properties:
           host:
             type: string
-            description: "Target hostname or IP address"
+			description: "Target hostname or IP address"
           ping_count:
             type: integer
-            description: "Number of pings to send"
-            default: 3
+			description: "Number of pings to send"
+            default: 4
           ping_interval:
             type: string
+			description: "Interval between pings"
+            format: duration
+            default: 300ms
+          ping_timeout:
+            type: string
+			description: "Timeout of ping request"
             format: duration
-            x-customType: "time.Duration"
-            description: "Interval between pings"
-            default: "1s"
-        required: ["host"]
+            default: 4s
+      required: [ host ]
+      minItems: 1
+    custom_field:
+      go:
+        custom: ['type', 'default', 'validate']
+  required: [ targets ]
+  allOf:
+    - $ref: 'go.opentelemetry.io/collector/scraper/scraperhelper.controller_config'
+    - $ref: './internal/metadata.metrics_builder_config' 
 ```
 
-`#/$defs/MetricsBuilderConfig` would be automatically generated by mdatagen with the same process used to generate the go structs and documentation today.
+#### Schema specification details
 
-`go.opentelemetry.io/collector/scraper/scraperhelper#/$defs/ControllerConfig` would be generated by the new tool from the schema definition in the scraperhelper component.
+- **format** of config schema follows roughly JSON Schema specification (https://json-schema.org/)
+- **special types** like time.Duration or time.Time can be represented as strings with specific formats (e.g., "duration", "date-time").
+- **dependencies** are specified with `$ref` attribute to reference other schema definitions, either from external packages or internal definitions.
+  To identify references we need full package paths (for external) or use relative path (for internal):
+    - `go.opentelemetry.io/collector/scraper/scraperhelper.controller_config` would be automatically generated by mdatagen with the same process used to generate the go structs and documentation today.
+    - `./internal/metadata.metrics_builder_config` would be generated by `mdatagen` from metrics definition in the scraperhelper component.
+    - dependencies is from external packages not covered with schemas will be replaced by `any` type with actual type annotation.
+- **default values** can be specified using the `default` attribute.
+- **validation** is possible using standard JSON Schema attributes, see [docs](https://json-schema.org/draft/2020-12/json-schema-validation).
 
 #### Extensibility
 
-The YAML schema specification can be extended with custom fields (e.g., `x-customType`) to capture domain-specific types and validation rules that are not natively supported in JSON schema. Additionally, we may introduce custom fields that generate fields that will produce references to structs or validation functions that require more complex logic and manual implementation.
+The YAML schema specification can be extended with custom annotations to capture domain-specific types and validation rules that are not natively supported in JSON schema. Additionally, we may introduce custom fields that generate fields that will produce references to structs or validation functions that require more complex logic and manual implementation.
+
+One proposal is to have `go` annotation that could contain the following sub-attributes:
+- type [string] - allows specifying a custom Go type for the field in a format `<package>.<type_name>`.
+- pointer [bool] - if set to true, the generated field will be a pointer to the specified type.
+- optional [bool] - if set to true, the generated field will be wrapped in Optional[...] generic type.
+- custom [list of strings] - allows specifying custom code snippets to be injected into the generated code.
+  The possible values are: `type`, `default`, `validate`.
+  Example:
+  ```yaml
+  some_optional_field:
+   type: string
+   go:
+     type: com.github/custom/custompkg.CustomType
+     pointer: true
+     optional: true
+     custom: ['validate']
+  ``` 
+
+### Generated Go code example
+
+See below for an example of the generated Go code from the above schema:
+
+```go
+package icmpcheckreceiver
+
+import (
+	"time"
+
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/scraper/scraperhelper"
+	"go.uber.org/multierr"
+
+	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/icmpcheckreceiver/internal/metadata"
+)
+
+type Config struct {
+	scraperhelper.ControllerConfig `mapstructure:",squash"`
+	metadata.MetricsBuilderConfig  `mapstructure:",squash"`
+	Targets                        []TargetsItemConfig `mapstructure:"targets"`
+	CustomField                    CustomFieldType     `mapstructure:"custom_field"`
+}
+
+// subtype for targets item generated from schema
+type TargetsItemConfig struct {
+	Host         string        `mapstructure:"host"`
+	PingCount    int           `mapstructure:"ping_count"`
+	PingInterval time.Duration `mapstructure:"ping_interval"`
+	PingTimeout  time.Duration `mapstructure:"ping_timeout"`
+}
+
+func (c *Config) Validate() error {
+	var err error
+
+	if len(c.Targets) == 0 {
+		return multierr.Append(err, errMissingTarget)
+	}
+
+	for _, target := range c.Targets {
+		if target.Host == "" {
+			err = multierr.Append(err, errMissingTargetHost)
+		}
+	}
+
+	err = multierr.Append(err, ValidateCustomField(c.CustomField))
+
+	return err
+}
+
+func createDefaultConfig() component.Config {
+	return &Config{
+		// default values derived from schema
+		CustomField: DefaultCustomFieldValue(),
+	}
+}
+
+```
+
+#### Generated code details
+
+- The generated `config.go` file will define Go structs that mirror the configuration schema. It's capable
+  of handling nested objects and arrays and creating subtypes with auto-generated names.
+- The `Validate` method will include validation logic based on the schema (e.g., required fields).
+- A `createDefaultConfig` function will be generated to provide default values for the configuration.
+
+#### Customization
+
+Because schema defined field with custom type, validation and default value called `custom_field`, generated code will refer to placeholders that must be implemented in component's Go package. 
+This creates an implementation hook for component developers to add the custom logic.
+
+In the above example:
+1. `CustomFieldType` - a Go type for a `custom_field`
+2. `ValidateCustomField` - a function that provides validation of `custom_field` values
+3. `DefaultCustomFieldValue` - a function that returns default value(s) for `custom_field`
+
+### Generated JSON Schema example
+
+This file will be a direct translation of the `config` section from `metadata.yaml` into JSON Schema format.
+
+Example:
+```json
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/icmpcheckreceiver",
+  "title": "receiver/icmpcheck component",
+  "$defs": {
+    "metadata.metric_config": {
+      "description": "MetricConfig provides common config for a particular metric.",
+      "type": "object",
+      "properties": {
+        "enabled": {
+          "type": "boolean"
+        }
+      }
+    },
+    "metadata.metrics_builder_config": {
+      "description": "MetricsBuilderConfig is a configuration for icmpcheckreceiver metrics builder.",
+      "type": "object",
+      "properties": {
+        "metrics": {
+          "$ref": "#/$defs/metadata.MetricsConfig"
+        },
+        "resource_attributes": {
+          "$ref": "#/$defs/metadata.ResourceAttributesConfig"
+        }
+      }
+    },
+    "metadata.metrics_config": {
+      "description": "MetricsConfig provides config for icmpcheckreceiver metrics.",
+      "type": "object",
+      "properties": { ... }
+    },
+    "metadata.resource_attribute_config": {
+      "description": "ResourceAttributeConfig provides common config for a particular resource attribute.",
+      "type": "object",
+      "properties": { ... }
+    },
+    "metadata.resource_attributes_config": { ... },
+    "scraperhelper.controller_config": {
+      "description": "ControllerConfig defines common settings for a scraper controller configuration. Scraper controller receivers can embed this struct, instead of receiver.Settings, and extend it with more fields if needed.",
+      "type": "object",
+      "properties": {
+        "collection_interval": {
+          "description": "CollectionInterval sets how frequently the scraper should be called and used as the context timeout to ensure that scrapers don't exceed the interval.",
+          "type": "string",
+          "pattern": "^(?:[0-9]+(\\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$",
+          "default": "60s"
+        },
+        "initial_delay": {
+          "description": "InitialDelay sets the initial start delay for the scraper, any non positive value is assumed to be immediately.",
+          "type": "string",
+          "pattern": "^(?:[0-9]+(\\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$",
+          "default": "1s"
+        },
+        "timeout": {
+          "description": "Timeout is an optional value used to set scraper's context deadline.",
+          "type": "string",
+          "pattern": "^(?:[0-9]+(\\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$",
+          "default": "0s"
+        }
+      }
+    }
+  },
+  "type": "object",
+  "properties": {
+    "targets": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "host": {
+            "type": "string"
+          },
+          "ping_count": {
+            "type": "integer",
+            "default": 4
+          },
+          "ping_interval": {
+            "type": "string",
+            "pattern": "^(?:[0-9]+(\\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$",
+            "default": "300ms"
+          },
+          "ping_timeout": {
+            "type": "string",
+            "pattern": "^(?:[0-9]+(\\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$",
+            "default": "4s"
+          }
+        },
+        "required": [
+          "host"
+        ],
+        "minItems": 1
+      },
+      "required": [
+        "targets"
+      ]
+    },
+    "custom_field": {
+      "type": {} // any
+    }
+  },
+  "allOf": [
+    {
+      "$ref": "#/$defs/scraperhelper.controller_config"
+    },
+    {
+      "$ref": "#/$defs/metadata.metrics_builder_config"
+    }
+  ]
+}
+```
+
+#### Generated JSON Schema details
+
+- The `config.schema.json` file will be a direct representation of the `config` section from `metadata.yaml`,
+  with proper handling of `$ref` references to include definitions from external packages and internal definitions
+  in a single JSON Schema file.
+- Each referenced schema will be embedded inline.
+- The `$id` field will be set to the component's import path for easy identification.
+- The custom types will be represented as `any` type in JSON Schema.
+
 
 ### Roadmap
 
@@ -132,3 +370,11 @@ If existing config structs don't follow the established naming patterns produced
 **Success Criteria:**
 - All core and contrib components migrated to schema-first approach
 - All new components use schema-first tooling by default
+
+#### Phase 4: Extend OCB
+
+**Objective**: Use OCB to bind all components schemas to single schema that describes entire collector config file
+
+**Success Criteria:**
+- Generated schema for collector config file includes all components
+- Schema can be used by internal and external tools for validation and autocompletion