fix: use safe default values for servicemonitor (#113)

Author: bsctl

charts/kamaji-etcd/README.md

@@ -100,14 +100,16 @@ Here the values you can override:
 | serviceAccount.name | string | `""` | Define the ServiceAccount name to use during the setup and provision of the etcd backing storage (default: "") |
 | serviceMonitor.annotations | object | `{}` | Assign additional Annotations |
 | serviceMonitor.enabled | bool | `false` | Enable ServiceMonitor for Prometheus |
-| serviceMonitor.endpoint | object | `{"interval":"15s","metricRelabelings":[],"relabelings":[],"scrapeTimeout":""}` | ServiceAccount for scraping metrics from etcd serviceAccount:   # -- ServiceAccount name   name: prometheus   # -- ServiceAccount namespace   namespace: monitoring-system |
 | serviceMonitor.endpoint.interval | string | `"15s"` | Set the scrape interval for the endpoint of the serviceMonitor |
 | serviceMonitor.endpoint.metricRelabelings | list | `[]` | Set metricRelabelings for the endpoint of the serviceMonitor |
 | serviceMonitor.endpoint.relabelings | list | `[]` | Set relabelings for the endpoint of the serviceMonitor |
 | serviceMonitor.endpoint.scrapeTimeout | string | `""` | Set the scrape timeout for the endpoint of the serviceMonitor |
-| serviceMonitor.labels | object | `{}` | Assign additional labels according to Prometheus' serviceMonitorSelector matching labels |
-| serviceMonitor.matchLabels | object | `{}` | Change matching labels |
-| serviceMonitor.namespace | string | `""` | Install the ServiceMonitor into a different Namespace, as the monitoring stack one (default: the release one) |
+| serviceMonitor.labels | object | `{"release":"kube-prometheus-stack"}` | Assign additional labels according to Prometheus' serviceMonitorSelector matching labels. By default, it uses the kube-prometheus-stack one. |
+| serviceMonitor.matchLabels | object | `{}` | Change matching labels. By default, it uses client service labels. |
+| serviceMonitor.namespace | string | `""` | Install the ServiceMonitor into a different namespace than release one. |
+| serviceMonitor.serviceAccount | object | `{"name":"kube-prometheus-stack-prometheus","namespace":"monitoring-system"}` | ServiceAccount for scraping metrics from etcd. By defult, it uses the kube-prometheus-stack one. |
+| serviceMonitor.serviceAccount.name | string | `"kube-prometheus-stack-prometheus"` | ServiceAccount name |
+| serviceMonitor.serviceAccount.namespace | string | `"monitoring-system"` | ServiceAccount namespace |
 | serviceMonitor.targetLabels | list | `[]` | Set targetLabels for the serviceMonitor |
 | snapshotCount | string | `"10000"` | Number of committed transactions to trigger a snapshot to disk. |
 | tolerations | list | `[]` | Kubernetes node taints that the etcd pods would tolerate |

charts/kamaji-etcd/values.yaml

@@ -137,22 +137,23 @@ datastore:
 serviceMonitor:
   # -- Enable ServiceMonitor for Prometheus
   enabled: false
-  # -- Install the ServiceMonitor into a different Namespace, as the monitoring stack one (default: the release one)
+  # -- Install the ServiceMonitor into a different namespace than release one.
   namespace: ''
-  # -- Assign additional labels according to Prometheus' serviceMonitorSelector matching labels
-  labels: {}
+  # -- Assign additional labels according to Prometheus' serviceMonitorSelector matching labels. By default, it uses the kube-prometheus-stack one.
+  labels:
+    release: kube-prometheus-stack
   # -- Assign additional Annotations
   annotations: {}
-  # -- Change matching labels
+  # -- Change matching labels. By default, it uses client service labels.
   matchLabels: {}
   # -- Set targetLabels for the serviceMonitor
   targetLabels: []
-  # -- ServiceAccount for scraping metrics from etcd
-  # serviceAccount:
-  #   # -- ServiceAccount name
-  #   name: prometheus
-  #   # -- ServiceAccount namespace
-  #   namespace: monitoring-system
+  # -- ServiceAccount for scraping metrics from etcd. By defult, it uses the kube-prometheus-stack one.
+  serviceAccount:
+    # -- ServiceAccount name
+    name: kube-prometheus-stack-prometheus
+    # -- ServiceAccount namespace
+    namespace: monitoring-system
   endpoint:
     # -- Set the scrape interval for the endpoint of the serviceMonitor
     interval: "15s"

docs/monitoring.md

@@ -16,32 +16,60 @@ spec:
 ...
 ```
 
-These metrics can be collected by Prometheus and used for both monitoring and debugging purposes.
+The `etcd` metrics are exposed on port `2381` by default, and can be accessed via the `/metrics` endpoint of kamaji-etcd-client service:
 
-## Setting up Prometheus
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    prometheus.io/metrics: "true"
+  name: kamaji-etcd-client
+spec:
+  ports:
+  - name: client
+    port: 2379
+    protocol: TCP
+    targetPort: 2379
+  - name: metrics
+    port: 2381
+    protocol: TCP
+    targetPort: 2381
+  selector:
+    app.kubernetes.io/instance: kamaji-etcd
+```
+
+These metrics can be collected by Prometheus Operator and used for both monitoring and debugging purposes.
 
-To begin monitoring `kamaji-etcd`, install the Prometheus stack using Helm:
+## Setting up Prometheus Operator
+
+To begin monitoring `kamaji-etcd`, install the Prometheus Operator stack using Helm:
 
 ```sh
-helm repo add bitnami https://charts.bitnami.com/bitnami
+helm repo add kube-prometheus-stack https://prometheus-community.github.io/helm-charts
 helm repo update
-helm install kube-prometheus bitnami/kube-prometheus \
+helm install kube-prometheus kube-prometheus-stack/kube-prometheus-stack \
     --set prometheus.persistence.enabled=true \
     --namespace monitoring-system \
     --create-namespace
 ```
 
-The `kamaji-etcd` Helm chart includes an optional `ServiceMonitor` for Prometheus, enabling it to scrape `etcd` metrics:
+The `kamaji-etcd` Helm chart includes an optional `ServiceMonitor` for Prometheus Operator, enabling it to scrape `etcd` metrics:
 
 ```sh
 helm -n kamaji-etcd upgrade kamaji-etcd clastix/kamaji-etcd \
     --set datastore.enabled=true \
     --set datastore.name=default \
-    --set fullnameOverride=default \
-    --set serviceMonitor.enabled=true \
-    --set serviceMonitor.namespace=monitoring-system
+    --set fullnameOverride=kamaji-etcd \
+    --set serviceMonitor.enabled=true
 ```
 
+By default such `ServiceMonitor` is installed in the same namespace as the `kamaji-etcd` release. If you want to install it in a different namespace, you can set the `serviceMonitor.namespace` value to the desired namespace.
+
+By default, the `ServiceMonitor` is configured to scrape metrics every 15 seconds. You can adjust this interval by setting the `serviceMonitor.endpoint.interval` value.
+
+By default, the `ServiceMonitor` uses the ServiceAccount `kube-prometheus-stack-prometheus` in the `monitoring-system` namespace to scrape metrics from etcd. If you want to use a different ServiceAccount, you can set the `serviceMonitor.serviceAccount.name` and value `serviceMonitor.serviceAccount.namespace` to the desired values.
+
 ## Key Metrics to Monitor
 
 Here’s a summary of important `etcd` metrics that help monitor cluster health. A full list of metrics is available [here](https://etcd.io/docs/v3.5/metrics/etcd-metrics-latest.txt).
@@ -97,7 +125,7 @@ max(rate(etcd_server_proposals_failed_total[60m]))
 High disk write latency can signal disk issues and affect `etcd` stability. To monitor disk performance, visualize the time taken to commit 99% of requests:
 
 ```sh
-histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket{job=~"etcd"}[5m])) by (le,instance))
+histogram_quantile(0.99, sum(rate(etcd_disk_backend_commit_duration_seconds_bucket{job=~"kamaji-etcd"}[5m])) by (le,instance))
 ```
 
 ### Network Metrics