[nr-k8s-otel-collector] make collector observability resource attributes configurable#2133
Conversation
|
|
|
@dbudziwojskiNR @Philip-R-Beckwith would love feedback on this at your earliest convenience. Thanks! |
| endpoint: {{ include "nrKubernetesOtel.endpoint" . }} | ||
| headers: | ||
| api-key: ${env:NR_LICENSE_KEY} | ||
| resource: |
There was a problem hiding this comment.
Could we stamp the cluster name through a traditional processor in a pipeline instead? This leaves the door open to stamping other attributes from processors like resource_detection
| endpoint: {{ include "nrKubernetesOtel.endpoint" . }} | ||
| headers: | ||
| api-key: ${env:NR_LICENSE_KEY} | ||
| resource: |
| # to the chart and its templates, including the app version. | ||
| # Versions are expected to follow Semantic Versioning (https://semver.org/) | ||
| version: 0.10.8 | ||
| version: 0.10.9 |
There was a problem hiding this comment.
You don't need to bump this version anymore, our CICD process will do so on release.
| scrapeIntervalMs: 60000 | ||
| # -- Specifies the interval at which the collector reports its metrics (in seconds) | ||
| # @default -- `60` | ||
| scrapeIntervalSeconds: 60 |
There was a problem hiding this comment.
This value should probably remain as scrapeIntervalMs, and instead we should fix this on the deployment and daemonset since periodic.interval accepts ms, not seconds. Alternatively, we could try and normalize this to seconds. Open to ideas.
What this PR does / why we need it:
Exposes
service::telemetry::resourceas a configurable Helm value undercollectorObservability.resource, allowing users to enrich internalotelcol_*metrics with custom resource attributes.Without this, internal collector metrics arrive in New Relic with no cluster or environment context, making it impossible to distinguish which cluster or environment a collector's internal metrics originate from in multi-cluster deployments. The
resource/newrelicprocessor correctly enriches pipeline metrics withk8s.cluster.name, butservice::telemetryexports directly via OTLP and bypasses all pipeline processors, so it never receives that enrichment.This PR adds a
resourceblock underservice::telemetryin both the daemonset and deployment configmaps.k8s.cluster.nameis automatically populated from the existingnewrelic.common.clusterhelper, following the same pattern already used in theresource/newrelicprocessor. Additional arbitrary attributes can be passed viacollectorObservability.resourcefor things likeenv.This also fixes a minor inconsistency where
values.yamldocumented the scrape interval field asscrapeIntervalMsbut the configmap template referenced it asscrapeIntervalSeconds.Which issue this PR fixes
(optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close that issue when PR gets merged)Special notes for your reviewer:
resourceblock is placed atservice::telemetry::resourceper the OTel collector internal telemetry docs, which applies resource attributes across all internal telemetry signals (metrics, logs, and traces)resourceblock is only rendered whencollectorObservability.enabledistrue, so there is no impact to users who have not enabled collector observabilityk8s.cluster.nameis always set automatically and does not need to be specified by the userdaemonset-configmap.yamlanddeployment-configmap.yamlscrapeIntervalMs→scrapeIntervalSecondsrename invalues.yamlis a documentation fix only — the configmap template already usedscrapeIntervalSecondsso behavior is unchangedChecklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]
[mychartname])Release Notes to Publish (nr-k8s-otel-collector)
If this PR contains changes in
nr-k8s-otel-collector, please complete the following section. All other charts should ignore this section.🚀 What's Changed
collectorObservability.resourceto allow users to enrich internal collector metrics with custom resource attributes.k8s.cluster.nameis automatically populated from the chart's cluster value, enabling multi-cluster observability out of the box. Additional attributes such asenvcan be passed viacollectorObservability.resource.scrapeIntervalMsinvalues.yamlbutscrapeIntervalSecondsin the configmap template.