You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
refactor(kubeaid-agent): fold the security exporter into this chart
Collapses argocd-helm-charts/kubeaid-security-exporter into the agent chart, so
a cluster gets both workloads from one Argo CD application instead of two, and
there is one chart to maintain rather than two.
They remain TWO Deployments with TWO ServiceAccounts. That is the point of the
arrangement, not an accident of it: a pod carries a single ServiceAccount, so
running the exporter as a sidecar would hand the workload holding the Obmondo
mTLS credential the exporter's cluster-wide read across seven API groups --
undoing, one layer down, the separation that splitting them achieved. It also
bounds failure: a collection pass holds every VulnerabilityReport in memory at
once, and a sidecar OOM would take the agent down with it, silencing the
cluster-liveness ping that tells the portal a cluster is alive.
The exporter's Service name is pinned to kubeaid-security-exporter rather than
derived from the release name. The agent's default
appConfig.securityPosture.exporterURL is that literal string, and a
release-prefixed Service would leave the agent polling a host that does not
resolve -- which fails quietly, because a failed poll only sets a metric and
submits nothing. The helper says so, so nobody "fixes" it later.
securityExporter.enabled: false runs the agent alone. The agent already
tolerates a missing exporter, so that switch and
appConfig.securityPosture.enabled stay independent.
backup-exporter is deliberately left as its own chart. Its values are some two
hundred lines of S3 credentials, RPO policy and four alert blocks for Postgres,
Velero, MongoDB and sealed-secrets, none of which the agent has anything to do
with; folding it in would make the agent chart mostly backup configuration, and
would drag five docs pages and a promtool CI job along with it.
Copy file name to clipboardExpand all lines: argocd-helm-charts/kubeaid-agent/README.md
+85-12Lines changed: 85 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,6 +4,23 @@ The KubeAid Agent (`ghcr.io/obmondo/kubeaid-agent`) is Obmondo's in-cluster agen
4
4
Obmondo API over mTLS and reports cluster state — node counts, Kubernetes version, security posture — and can run
5
5
Argo CD syncs during agreed service windows. This chart is authored in KubeAid (not a vendored upstream wrapper).
6
6
7
+
## What this chart deploys
8
+
9
+
Two Deployments from one Argo CD application:
10
+
11
+
| Workload | Purpose |
12
+
|---|---|
13
+
|`kubeaid-agent`| Talks to the Obmondo API over mTLS. Holds the credential; holds no CRD access. |
14
+
|`kubeaid-security-exporter`| Collects the cluster's security posture and serves it at `/api/v1/security-posture`. Holds cluster-wide read; talks to nothing outside the cluster. |
15
+
16
+
They are **two Deployments with two ServiceAccounts, not one pod with a sidecar**. A pod carries a single
17
+
ServiceAccount, so co-locating them would hand the workload holding the Obmondo credential the exporter's
18
+
cluster-wide read across seven API groups — the coupling that separating them removed in the first place.
19
+
Keeping them apart also bounds the blast radius: a collection pass holds every VulnerabilityReport in memory
20
+
at once, and as a sidecar an OOM there would take down the agent, and with it the cluster-liveness ping.
21
+
22
+
Set `securityExporter.enabled: false` to run the agent alone.
23
+
7
24
## Why it's in KubeAid
8
25
9
26
It is the link between a KubeAid cluster and the Obmondo platform. On clusters with `obmondo.monitoring` enabled,
@@ -18,9 +35,16 @@ on the `kubeaid` Argo CD project, stored in the `argocd-project-role-kubeaid-age
18
35
-`argocd-project-role-kubeaid-agent` Secret in the `argocd` namespace, holding the Argo CD auth token under the
19
36
`token` key (created automatically by `kubeaid-cli`). Name overridable via
20
37
`appConfig.argocd.authTokenSecretName`.
21
-
-`kubeaid-security-exporter` in the cluster, if `appConfig.securityPosture.enabled` stays `true`. The agent
22
-
forwards that exporter's snapshots; it collects no posture data itself.
23
-
- kube-prometheus, if `serviceMonitor` stays enabled (defaults to `true`).
38
+
- kube-prometheus, if `serviceMonitor` / `securityExporter.prometheusRule` stay enabled (all default to `true`).
39
+
40
+
The exporter additionally wants, but does not require:
41
+
42
+
-**A vulnerability scanner** — either **trivy-operator** or **kubescape-operator**. Without one it reports no
43
+
findings, and says so rather than reporting a clean cluster.
44
+
-**version-checker**, for upgrade availability. Without it findings still ship, with upgrade availability
45
+
unknown rather than "up to date".
46
+
47
+
Cilium, Tetragon and KubeArmor are read when present and skipped when not.
24
48
25
49
## Key values / KubeAid-specific configuration
26
50
@@ -32,22 +56,71 @@ on the `kubeaid` Argo CD project, stored in the `argocd-project-role-kubeaid-age
32
56
|`appConfig.obmondoAPI.url`|`https://api.obmondo.com/api`| Obmondo API endpoint (mTLS). |
33
57
|`appConfig.kubeaidUpdate.enabled`|`false`| Opt-in: schedule the service-window Argo CD sync cron job. |
34
58
|`appConfig.kubeaidUpdate.checkInterval`|`15m`| Poll cadence for an active KubeAid update service window. |
35
-
|`appConfig.securityPosture.enabled`|`true`| Poll `kubeaid-security-exporter` and forward its snapshots to the Obmondo API. The agent collects nothing itself. |
36
-
|`appConfig.securityPosture.exporterURL`|`http://kubeaid-security-exporter`| In-cluster URL of the exporter. A bare Service name resolves in the agent's own namespace; qualify it if the two charts deploy to different namespaces. |
37
-
|`appConfig.securityPosture.pollInterval`|`1h`| Poll cadence. The submit is skipped when `collectedAt` has not advanced, so end-to-end freshness is bounded by the exporter's collection interval, not by this. |
59
+
|`appConfig.securityPosture.enabled`|`true`| Poll the exporter and forward its snapshots to the Obmondo API. The agent collects nothing itself. |
60
+
|`appConfig.securityPosture.exporterURL`|`http://kubeaid-security-exporter`| In-cluster URL of the exporter. Matches the Service this chart creates — change both or neither. |
61
+
|`appConfig.securityPosture.pollInterval`|`1h`| Poll cadence. The submit is skipped when `collectedAt` has not advanced, so end-to-end freshness is bounded by `securityExporter.exporter.interval`, not by this. |
38
62
|`obmondoAPITLSSecretName`|`obmondo-clientcert`| Secret with the mTLS keypair. |
39
63
|`extraSecretReaderNamespaces`|`[]`| Extra namespaces where a secrets-read Role/RoleBinding is created for the agent. |
64
+
|`securityExporter.enabled`|`true`| Deploy the exporter alongside the agent. |
65
+
|`securityExporter.exporter.interval`|`12h`| Collection cadence. Trivy refreshes its reports on a 24h TTL, so polling faster re-reads identical data. |
66
+
|`securityExporter.prometheusRule.upgradableThreshold`|`20`|`ImageOutdatedAndVulnerable` fires above this many images having both a fixable Critical/High CVE and a newer tag available. |
67
+
|`securityExporter.prometheusRule.upgradableFor`|`24h`| How long the count must hold before the alert fires. |
68
+
69
+
## What the exporter collects
70
+
71
+
Each source is detected through API discovery and skipped when absent.
72
+
73
+
-**Vulnerabilities and least-privilege findings** from Trivy Operator's report CRs — full CVE detail
74
+
(CVSS score, installed and fixed version, advisory link), rather than the lossy Prometheus projection.
75
+
-**Or the same from Kubescape**, read through its aggregated APIService, with per-CVE relevancy — whether the
76
+
vulnerable code actually loaded. The two scanners are alternatives, never merged.
77
+
-**Upgrade availability**, by joining findings against version-checker on canonical image references resolved
78
+
in Go. Rebuilding image references in PromQL fails silently.
79
+
-**Network enforcement** from Cilium — whether policy is actually realised on an app's pods, which is a
80
+
different question from whether the app ships a policy.
81
+
-**Runtime detection posture** from Tetragon and KubeArmor — which engines are armed, and whether they
82
+
observe or enforce.
83
+
84
+
### Which scanner is used
85
+
86
+
Exactly one, chosen by API discovery: **Trivy takes precedence, Kubescape is used where Trivy is absent.**
87
+
Merging them would double-count the same CVE from two databases that disagree at the margins, and switching an
88
+
existing cluster's scanner rewrites every finding's ID, score and link at once — which reads as mass CVE churn
89
+
rather than as a configuration change. The snapshot names the one it used in `scanner`.
90
+
91
+
Two fields differ by scanner, and both are absent rather than false when unanswerable:
92
+
93
+
| Field | Trivy | Kubescape |
94
+
|---|---|---|
95
+
|`relevant` (did the vulnerable code load) | never — Trivy cannot observe runtime | when the eBPF node-agent runs |
96
+
|`os.eosl` (base image past end of life) | yes | never — Grype has no end-of-life data |
97
+
98
+
## Alerting
99
+
100
+
One alert, `ImageOutdatedAndVulnerable`. It is a count, so it fires once per cluster rather than once per
101
+
image, and the threshold is deliberately high — every real cluster carries a few of these at any moment, so a
102
+
low threshold fires everywhere on day one and gets ignored. The signal worth acting on is a pile of easy
103
+
upgrades, not the existence of one.
104
+
105
+
Collection status is exported as `security_exporter_collection` (1 ok, 0 failed, -1 not installed) but is
106
+
deliberately not alerted on: a collection failure is a debugging signal, not something worth paging for.
40
107
41
108
## Operational notes
42
109
43
-
- RBAC is least-privilege by construction: a purpose-built ClusterRole grants exactly the verbs the agent's code
44
-
calls (no `watch`, no blanket `view`). Secrets access stays on namespaced Roles. The agent holds no CRD access
45
-
at all — reading Trivy, Cilium, Tetragon and KubeArmor resources belongs to `kubeaid-security-exporter`.
46
-
-`securityPosture.enabled: true` is safe where the exporter is not installed — the poll fails, a metric records
47
-
it, and nothing is submitted. Set it to `false` only where vulnerability detail must not leave the cluster.
48
-
- Runs unprivileged: non-root, all capabilities dropped, `RuntimeDefault` seccomp.
110
+
- RBAC is least-privilege by construction, and **separate per workload**: the agent gets a purpose-built
111
+
ClusterRole granting exactly the verbs its code calls, with secrets access on namespaced Roles and no CRD
112
+
access at all. The exporter gets its own, covering the report CRs it reads. Neither can use the other's.
113
+
- No `watch` anywhere — neither workload builds informers.
114
+
- The exporter's rules are all consumed through a dynamic client, so no Go source references those resource
115
+
names; removing a rule fails silently and renders the cluster as clean rather than erroring.
116
+
-`appConfig.securityPosture.enabled: true` is safe where the exporter is disabled — the poll fails, a metric
117
+
records it, and nothing is submitted. Set it to `false` only where vulnerability detail must not leave the
118
+
cluster.
119
+
- Both run unprivileged: non-root, all capabilities dropped, `RuntimeDefault` seccomp; the exporter also runs
120
+
with a read-only root filesystem.
49
121
50
122
## Docs links
51
123
52
124
- Chart source: `templates/` and [values.yaml](./values.yaml) in this directory (documented inline).
0 commit comments