Kubernetes Grafana Dashboards to OpenTelemetry Collector (DaemonSet) Metrics#2094
Kubernetes Grafana Dashboards to OpenTelemetry Collector (DaemonSet) Metrics#2094ThashmikaX wants to merge 13 commits intoopen-telemetry:mainfrom
Conversation
|
Many thanks @ThashmikaX. Can you please share with us the OpenTelemetry Collector configuration you use with these dashboards? Do these dashboards rely on metrics produced by the OTelCollector KubeletStatsReceiver, the k8sclusterreceiver, and the hostmetricsreceiver? The dashboards of this PR seem to often use Prometheus style instrumentation metrics like |
|
FYI we have started documenting the mapping from Prometheus metrics used in the kubernetes-mixin to their OTel equivalent in [kubernetes-mixin/docs/otel-mapping.md}(https://github.com/kubernetes-monitoring/kubernetes-mixin/blob/master/docs/otel-mapping.md) In parallel we work at porting the Kubernetes Mixin to OTel metrics on grafana/kubernetes-mixin-otel which is intended to move to the Github org https://github.com/kubernetes-monitoring once it gets more stable, contributions are welcome. Waiting for @ThashmikaX 's answer on the metrics consumed by the dashboards of this PR, we will continue to work on the mixin repos. |
Thanks for pointing that out. Yes, the dashboards are generally intended to work with metrics coming from the kubeletstatsreceiver, k8sclusterreceiver, and hostmetricsreceiver. In a few places I’ve used Prometheus-style metrics (e.g., container_cpu_usage_seconds_total) because those are what we currently have available in our environment. I can align everything to the OTel semantic metrics. Will do the changes to this PR. Will share the OpenTelemetry Collector configuration. |
|
|
Thanks for the clarification @ThashmikaX ! |
|
I will update the dashboards to consistently use the OTel semantic metrics and push the changes to this PR. |
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
|
Thanks @ThashmikaX , it's great to see more and more dashboard panels showing metrics! |
|
In the |
…cs and improve data representation
…cs and improve data representation
… and enhance data representation
…cs and improve legend formatting
…cs and improve data representation
- Updated the Kubernetes Kubelet dashboard to use OTel metrics for node, pod, and container counts, replacing previous kubelet metrics. - Changed expressions to count nodes, pods, containers, and volumes based on OTel metrics. - Renamed titles and descriptions for clarity and accuracy. - Adjusted the Node Exporter dashboard to utilize OTel metrics for CPU, memory, and disk usage, enhancing compatibility with Kubernetes environments. - Improved variable queries for clusters and instances to align with OTel metrics. - General cleanup and formatting improvements for better readability and maintainability.
|
Hi @cyrille-leclerc, can you check back again now? |
|
Thanks, I'll test ASAP. |
Port Kubernetes Grafana Dashboards to OpenTelemetry Collector (DaemonSet) Metrics
Summary
This PR ports several Kubernetes Grafana dashboards originally designed for
kube-prometheus-stack(Prometheus Node Exporter) to instead use metrics collected by the OpenTelemetry Collector running as a DaemonSet.Background / Motivation
As part of our observability migration, we transitioned from Prometheus Node Exporter (via
kube-prometheus-stack) to the OpenTelemetry Collector for node-level metrics collection. While this provides a more unified and flexible telemetry pipeline, it also results in differences in metric names and semantic conventions.Because of these differences, the default Grafana dashboards bundled with
kube-prometheus-stackno longer function correctly without modification.To preserve equivalent monitoring visibility and avoid losing operational insight, we have:
What This PR Includes
Community Value
These dashboards may be useful for teams:
kube-prometheus-stackto OpenTelemetry CollectorNotes
Hi @cyrille-leclerc, this PR is related to the dashboards mentioned here: Thread