Skip to content

Commit 4d0ee67

Browse files
authored
Merge branch 'main' into feat/aks-ib-rdma-via-nodewright
2 parents b870c32 + 713f263 commit 4d0ee67

4 files changed

Lines changed: 603 additions & 27 deletions

File tree

docs/conformance/cncf/index.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,14 @@ names are `dra`, `gang`, `secure`, `accelerator-metrics`, `service-metrics`,
9595
> type (NIM inference, Dynamo inference, or Kubeflow training) and collects
9696
> appropriate metrics and operator evidence.
9797
98+
Each section writes exactly one **PASS**, **SKIP**, or **FAIL** verdict.
99+
**SKIP** marks an absent optional prerequisite (no inference gateway, no
100+
supported operator, an unsupported cluster-autoscaling provider, or an
101+
operator present with no workload to reconcile) and is not a failure. A
102+
**FAIL** — an unhealthy present capability, a failed query, or a missing/
103+
malformed verdict — fails closed and makes the collection exit non-zero, so a
104+
recorded section failure is no longer masked as overall success.
105+
98106
### Two Modes
99107

100108
| | `aicr validate --phase conformance` | `--cncf-submission` |

docs/user/validation.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -536,6 +536,22 @@ Valid feature names (from `pkg/evidence/cncf/collector.go`):
536536
| `pod-autoscaling` | HPA-driven pod autoscaling: external GPU metric + behavioral scale-up/down test (pod-scoped custom metrics collected best-effort — absent for DRA-allocated GPUs, not a failure) |
537537
| `cluster-autoscaling` | Karpenter (preferred) or EKS managed node-group autoscaling fallback |
538538

539+
### Evidence verdicts and exit status
540+
541+
Each collected feature records exactly one verdict in its evidence file:
542+
543+
- **PASS** — the capability was exercised and behaved as expected.
544+
- **SKIP** — an optional prerequisite is absent (e.g. no inference gateway,
545+
no supported operator, an unsupported cluster-autoscaling provider, or an
546+
operator that is installed but has no workload to reconcile). A SKIP is not
547+
a failure.
548+
- **FAIL** — a present capability was unhealthy, a required query failed, or
549+
the evidence file carried no single valid verdict (fail closed).
550+
551+
A **FAIL** in any feature makes `--cncf-submission` exit non-zero, so a
552+
submission that records a genuine failure no longer reports overall success.
553+
Absent optional prerequisites remain successful SKIPs and do not fail the run.
554+
539555
## Emitting recipe evidence
540556

541557
When a recipe PR targets hardware AICR maintainers cannot independently

0 commit comments

Comments
 (0)