Skip to content

Node scan pods hit /proc access denied on non-OpenShift clusters (dropped capabilities) #1567

Description

@AdamVB

Summary

Node scanning pods on non-OpenShift clusters (e.g. EKS) log a large number of access denied errors while walking the host /proc. This is caused by the pod's SecurityContext dropping all Linux capabilities while relying on running as root (UID 0) to read /proc.

Root cause

The node scan pod SecurityContext is generated by the operator in controllers/nodes/resources.go (both the CronJob and DaemonSet paths):

SecurityContext: &corev1.SecurityContext{
    AllowPrivilegeEscalation: ptr.To(isOpenshift),
    ReadOnlyRootFilesystem:   ptr.To(true),
    RunAsNonRoot:             ptr.To(false),
    RunAsUser:                ptr.To(int64(0)),
    Capabilities: &corev1.Capabilities{
        Drop: []corev1.Capability{"ALL"},
    },
    // RHCOS requires to run as privileged to properly do node scanning. If the container
    // is not privileged, then we have no access to /proc.
    Privileged: ptr.To(isOpenshift),
},

On a non-OpenShift cluster isOpenshift == false, so:

  • Privileged: false
  • Capabilities.Drop: ["ALL"] — which includes CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, and CAP_SYS_PTRACE.

The container runs as UID 0, but root only bypasses DAC checks and can read other processes' /proc entries because it holds those capabilities. Once all capabilities are dropped, the kernel enforces permission checks against root like any other user. cnspec (MONDOO_PROCFS=on) then walks the host /proc mounted at /mnt/host/proc and hits EACCES/EPERM on /proc/<pid>/… entries owned by other processes/users (e.g. environ, fd, maps, cwd, root):

  • reading another process's environ/mem/fd requires CAP_SYS_PTRACE
  • traversing/reading files whose mode would otherwise deny root requires CAP_DAC_READ_SEARCH / CAP_DAC_OVERRIDE

The privileged escape hatch that would restore /proc access is currently gated solely on isOpenshift, so it never applies on non-OpenShift clusters.

Impact

  • Noisy access denied errors in node scan logs.
  • Incomplete /proc / process inventory data for node scans on non-OpenShift clusters, which may affect policy results that depend on process information.

Reproduction

  1. Install the operator on a non-OpenShift cluster (e.g. EKS).
  2. Enable node scanning.
  3. Inspect a node scan pod/job logs — observe repeated access denied errors referencing /proc.

Proposed fix

Prefer least privilege over flipping Privileged on everywhere. Options, in order of preference:

  1. Add only the capabilities needed for /proc — grant CAP_DAC_READ_SEARCH (and CAP_SYS_PTRACE if per-process detail such as environ/fd is required) via Capabilities.Add instead of dropping ALL with no additions. This restores /proc access without going fully privileged.
  2. Optionally expose this behavior behind a toggle on the MondooAuditConfig Nodes spec for users who want to opt in/out.
  3. Run privileged on non-OpenShift as a last resort (broadest access, largest security regression — not preferred).

References

  • controllers/nodes/resources.go — CronJob path (~L111) and DaemonSet path (~L245)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions