feat: address review feedback

Author: lalitadithya

docs/METRICS.md

@@ -168,7 +168,7 @@ submitting health events naming another node. See
 |------------|------|--------|-------------|
 | `platform_connector_auth_decisions_total` | Counter | `decision` | Health event batches by the node scope granted to the caller. Values: `node_local`, `cross_node` |
 | `platform_connector_auth_node_claim_total` | Counter | `result` | Whether an authenticated caller's token carried a node claim. `verified`: it carried one and it named this node. `absent`: it carried none, so the caller was pinned to this node — the same scope a tokenless caller gets. |
-| `platform_connector_auth_violations_total` | Counter | `reason` | Health event batches that violated the node-binding rule. Under `global.platformConnectorAuth.mode: enforce` (default) these are rejected; under `mode: audit` they are recorded here but let through. Reasons: `node_mismatch` (a node-local caller named a different node), `node_claim_mismatch` (the token was issued on another node — applies to every caller, allowlisted or not), `unbound_cross_node_token` (an allowlisted caller presented a token bound to no pod, e.g. from `kubectl create token`), `cross_node_claim_absent` (pod-bound, but the pod was never scheduled), `missing_node_name` (no node name on the event and none could be stamped), `token_invalid` (TokenReview rejected the token), `malformed_credentials` (the authorization header was duplicated or did not use the Bearer scheme). Three further reasons mean **no verdict could be reached**, not that the caller was rejected: `validator_unavailable` (the API server was unreachable), `validator_timeout` (the caller gave up or its deadline passed) and `validator_error` (an unexpected failure, typically this component's own RBAC to create TokenReviews). With `failOpenOnUnavailable: true`, `validator_unavailable` and `validator_timeout` still increment this counter but fall back to node-local scope instead of rejecting the request, in enforce mode too — an outage says nothing about the caller's credential. Exclude those three when alerting on suspected credential abuse — a control-plane outage increments them for every in-flight request and would otherwise look identical to an attack: `sum(rate(platform_connector_auth_violations_total{reason=~"node_mismatch\|token_invalid\|node_claim_mismatch\|unbound_cross_node_token\|malformed_credentials\|cross_node_claim_absent\|missing_node_name"}[5m]))` |
+| `platform_connector_auth_violations_total` | Counter | `reason` | Health event batches that violated the node-binding rule. Under `global.platformConnectorAuth.mode: enforce` (default) these are rejected; under `mode: audit` they are recorded here but let through. Reasons: `node_mismatch` (a node-local caller named a different node), `node_claim_mismatch` (the token was issued on another node — applies to every caller, allowlisted or not), `unbound_cross_node_token` (an allowlisted caller presented a token bound to no pod, e.g. from `kubectl create token`), `cross_node_claim_absent` (pod-bound, but the pod was never scheduled), `missing_node_name` (no node name on the event and none could be stamped), `token_invalid` (TokenReview rejected the token), `malformed_credentials` (the authorization header was duplicated or did not use the Bearer scheme). Three further reasons mean **no verdict could be reached**, not that the caller was rejected: `validator_unavailable` (the API server was unreachable), `validator_timeout` (the caller gave up or its deadline passed) and `validator_error` (an unexpected failure, typically this component's own RBAC to create TokenReviews). With `failOpenOnUnavailable: true`, `validator_unavailable` and `validator_timeout` still increment this counter but fall back to a degraded node-local scope instead of rejecting the request, in enforce mode too — an outage says nothing about the caller's credential. That scope is a guess, not a verified identity: an event with a blank or matching node name is accepted and stamped as usual, but one naming a different node is refused as retryable `Unavailable` rather than counted as `node_mismatch`, since the caller might really be an allowlisted cross-node publisher the outage prevented from being verified. `missing_node_name` is the one violation `mode: audit` does not let through — a blank node name from a cross-node caller is always rejected, in both modes, because nothing downstream can handle it. Exclude those three when alerting on suspected credential abuse — a control-plane outage increments them for every in-flight request and would otherwise look identical to an attack: `sum(rate(platform_connector_auth_violations_total{reason=~"node_mismatch\|token_invalid\|node_claim_mismatch\|unbound_cross_node_token\|malformed_credentials\|cross_node_claim_absent\|missing_node_name"}[5m]))` |
 
 ### Kubernetes Connector Metrics
 

docs/configuration/authentication.md

@@ -50,6 +50,13 @@ validator failure (including `validator_unavailable` and
 request. `audit` changes what happens after a violation is detected, not
 whether requests are checked.
 
+One rejection is not affected by `mode`: a cross-node caller's event with no
+node name (`missing_node_name`) is always rejected, in both `audit` and
+`enforce`. Nothing downstream can handle an empty node name, so forwarding
+that event would not be a useful preview of what `enforce` would do — it
+would just be an event `enforce` could never have produced, sitting in the
+datastore.
+
 ### `failOpenOnUnavailable`
 
 Distinguishes a validator that never reached a verdict from one that reached
@@ -62,9 +69,22 @@ credential) does say something about the caller and is always rejected under
 
 Defaults to `false`: an unreachable validator rejects the request, matching
 behaviour prior to this setting's introduction. Set `true` to fall back to
-node-local scope instead — the same treatment an anonymous caller gets —
+a **degraded** node-local scope instead — a guess, not a verified identity —
 so that a control-plane blip degrades publishers to their own node rather
-than blocking their health events outright.
+than blocking their health events outright:
+
+- An event with a blank or matching node name is accepted and stamped exactly
+  as a verified node-local caller's would be. This is the common case the
+  setting exists for: most publishers present a token but leave the node name
+  blank for platform-connector to fill in.
+- An event naming a *different* node is refused as retryable `Unavailable` —
+  the same code the validator itself returned — rather than as
+  `node_mismatch` / `PermissionDenied`. The caller might really be an
+  allowlisted cross-node publisher the outage prevented from being verified,
+  not an actual mismatch, so it is not counted as `node_mismatch` (one of the
+  reasons an operator would alert on as suspected credential abuse) and not
+  rejected in a way publishers treat as non-retryable; it is retried once the
+  validator recovers instead of being dropped for good.
 
 ### `audience`
 
@@ -156,7 +176,7 @@ cluster already uses.
 | `missing_node_name` | An event carried no node name and none could be stamped. |
 | `token_invalid` | TokenReview rejected the token. |
 | `malformed_credentials` | The authorization header was duplicated, or did not use the Bearer scheme. A *completely absent* header is not a violation — that caller is accepted and pinned to the connector's node. |
-| `validator_unavailable` / `validator_timeout` / `validator_error` | The API server could not be reached, or returned no identity. With `failOpenOnUnavailable: true`, `validator_unavailable` and `validator_timeout` still increment this counter but fall back to node-local scope instead of rejecting the request. |
+| `validator_unavailable` / `validator_timeout` / `validator_error` | The API server could not be reached, or returned no identity. With `failOpenOnUnavailable: true`, `validator_unavailable` and `validator_timeout` still increment this counter but fall back to a degraded node-local scope instead of rejecting the request — see [`failOpenOnUnavailable`](#failopenonunavailable) for how that scope treats a blank vs. a differently-named node. |
 
 A healthy cluster reports zero violations. A sustained non-zero
 `validator_unavailable` is an API-server problem, not a caller problem.

docs/platform-connectors.md

@@ -280,9 +280,15 @@ authenticated caller stays accepted for up to two minutes after its token
 expires or its pod is deleted. An unreachable API server is
 **retried inside the call** with exponential backoff for a few seconds before
 anything is surfaced. Past that the connector **fails closed** with
-`Unavailable`, which every publisher treats as retryable; tokenless callers
-never trigger a TokenReview at all, so custom monitors keep working with no
-API-server dependency whatsoever.
+`Unavailable` by default, which every publisher treats as retryable.
+`global.platformConnectorAuth.failOpenOnUnavailable: true` changes this: the
+caller falls back to a degraded node-local scope instead of being rejected —
+an outage says nothing about the caller's credential — with a blank or
+matching node name accepted and stamped as usual, and an event naming a
+different node kept retryable rather than silently accepted or counted as a
+mismatch. See [Authentication Configuration](configuration/authentication.md)
+for the detail. Tokenless callers never trigger a TokenReview at all, so
+custom monitors keep working with no API-server dependency whatsoever.
 
 #### Configuration
 
@@ -296,10 +302,15 @@ allowlist and the publishers' projected tokens cannot drift apart:
 | `tokenExpirationSeconds` | `3600` | Token lifetime. Must be a whole number in `600 <= x <= 2^32`; Kubernetes rejects anything else. |
 | `tokenMountPath` | `/var/run/secrets/nvsentinel/platform-connector` | Where publishers mount the token. |
 | `crossNodeServiceAccounts` | `[]` | **Additional** fully-qualified usernames allowed to name other nodes. The bundled monitors are derived from the release namespace for whichever are enabled — do not list them here. |
+| `mode` | `enforce` | `enforce` rejects a violating request. `audit` validates every request and still increments `platform_connector_auth_violations_total` by reason, but lets the request through instead of rejecting it — use it to roll node-binding out against real traffic, confirm the violation counters stay at zero, then switch to `enforce` with evidence. One exception applies in both modes: a cross-node caller's event with no node name is always rejected, since nothing downstream can handle it. |
+| `failOpenOnUnavailable` | `false` | When `true`, a validator that never reached a verdict (the API server was unreachable, or the call timed out) falls back to a degraded node-local scope instead of rejecting the request, since an outage says nothing about the caller's credential. A rejected credential (an invalid or malformed token) is always rejected regardless of this setting. |
 
-There is no `mode` and no `requireNodeClaim`. A setting that is "on but
-not enforcing" cannot be reasoned about from its configuration alone, so
-enforcement is not separable from enablement.
+There is no `requireNodeClaim`: whether a claim is checked is not
+configurable, only what happens after a violation is found (`mode`) and how an
+unavailable validator is treated (`failOpenOnUnavailable`). See
+[Authentication Configuration](configuration/authentication.md) for the full
+detail on both, including how a degraded scope treats a blank vs. a
+differently-named node.
 
 A **custom cluster-scoped monitor** needs three things: a projected token volume
 with the configured audience, its token path passed to the client dial, and its
@@ -384,7 +395,7 @@ Related metrics:
 | Metric | Labels | Meaning |
 |--------|--------|---------|
 | `platform_connector_auth_decisions_total` | `decision` | Batches by granted scope (`node_local`, `cross_node`) |
-| `platform_connector_auth_violations_total` | `reason` | Batches rejected by the interceptor |
+| `platform_connector_auth_violations_total` | `reason` | Batches that violated the node-binding rule. Rejected under `mode: enforce`; recorded but let through under `mode: audit`, except `missing_node_name`, which is always rejected. |
 | `platform_connector_auth_node_claim_total` | `result` | Whether the token carried a node claim (`verified`, `absent`) |
 
 Not every `reason` means a caller was rejected. `validator_unavailable`,