Add GB300 recipe overlays for EKS

[yuanchen8911]

Goal

Add service-bound recipe overlays for GB300 on EKS so that aicr recipe --accelerator gb300 --service eks --os ubuntu --intent <training|inference> ... resolves end to end, with a bundle that deploys and validates on a real GB300 NVL72 EKS cluster.

gb300 is already a declared accelerator in pkg/recipe/criteria.go, but recipes/overlays/ still has zero GB300 overlays, so every GB300 query fails to resolve.

Motivation

A GB300 EKS cluster is now available for development and qualification (nhensley-gb300, us-east-2):

• 2 x p6e-gb300r.36xlarge GPU nodes, Ubuntu 24.04, kernel 6.17.0-1019-aws, containerd 2.2.1
• 3 x m7i.xlarge system nodes (Amazon Linux 2023)
• Kubernetes v1.35.6
• Scale-out fabric is EFA (efa-only ENI on the GPU nodes), not RoCE — same fabric family as the existing GB200 EKS overlays
• topology.k8s.aws/network-node-layer-{1,2,3} labels present for topology-aware scheduling

This unblocks the overlay work that #1318 was closed as not-planned for (no hardware at the time), and it is the platform half of the ask in #1883 (GB300 NVL72 inference recipe referenced by the DSX NCP docs).

Scope

1. gb300-any.yaml cross-cutting overlay carrying the deployment-phase floor and component version pins, modeled on gb200-any.yaml.
2. Service-bound overlays for EKS, modeled on the gb200-eks-* family:
   • gb300-eks-training.yaml / gb300-eks-ubuntu-training.yaml
   • gb300-eks-inference.yaml / gb300-eks-ubuntu-inference.yaml
   • platform variants as warranted (-kubeflow, -inference-dynamo)
3. Machine-type qualification comments naming p6e-gb300r.36xlarge, consistent with the convention added for the other accelerators.
4. EFA + MNNVL/IMEX configuration: EFA device plugin (or networking DRA driver, per EFA on AWS: legacy device plugin vs networking DRA driver — should we migrate? #1326), NVLink compute-domain handling for GB300 NVL72.
5. Performance-phase thresholds tuned per leaf for the GB300 EFA fabric rather than inherited from GB200.
6. Cover the new overlays in recipes/overlays_validation_floor_test.go.
7. Run make bom-docs and commit the regenerated docs/user/container-images.md.

Success criteria

1. aicr recipe --accelerator gb300 --service eks --os ubuntu --intent training and the inference equivalent both resolve without manual overlay authoring.
2. aicr bundle output deploys cleanly on the GB300 EKS cluster.
3. aicr validate passes the deployment, conformance, and performance phases on that cluster.
4. New overlays are covered by the validation-floor test and make qualify is green.

Notes and open items

• GB300 inference for the DSX NCP use case is tracked separately in Add a GB300 NVL72 inference recipe #1883; this issue covers the EKS overlay family that inference variant would build on. Close or re-scope Add a GB300 NVL72 inference recipe #1883 once the inference leaf lands.
• The GPU operator and EFA device plugin are not yet installed on the cluster — no node advertises nvidia.com/gpu or vpc.amazonaws.com/efa today. Bringing those up is part of the first deploy.
• Each GPU node currently exposes a single EFA ENI on network card 0. Confirm whether that is the intended provisioning for p6e-gb300r.36xlarge before setting performance thresholds, since it caps scale-out bandwidth.
• Internal-only GB300 variants live in the internal recipes repo; keep the public overlays aligned rather than duplicating.

Related: #1883, #1318 (closed not-planned), #1326, #1410.

[yuanchen8911]

Cross-referencing #1883, which this addresses.

#1883 is the customer-facing report: NVIDIA DSX documentation ("Inference Provider Platform Requirements on GB300 NVL72 for NCPs") instructs NCPs to prepare Kubernetes with an AICR GB300 inference recipe, and aicr recipe --accelerator gb300 --os ubuntu --intent inference --platform dynamo does not resolve. It is on the v1 milestone.

The reason it stalled is the blocker this issue clears. Per the triage on #1883: the GB300 overlays landed in #1319 and were reverted in #1328 because hardware validation was not possible, and AICR publishes only recipes validated on real clusters — with a commitment to re-file upstream "once validation is possible." The nhensley-gb300 cluster described above is that opportunity.

Two things that follow for scope:

• The inference + Dynamo leaf is the one Add a GB300 NVL72 inference recipe #1883 needs. If training lands first, Add a GB300 NVL72 inference recipe #1883 stays open until that leaf ships.
• Scope item 3 (machine-type qualification comments naming p6e-gb300r.36xlarge) depends on the convention in docs(user): document the machine type each accelerator criterion targets #2380, still in review. Once that merges, the qualified-machine-types table in docs/user/cli-reference.md will also want a gb300 / eks row — most naturally added here rather than as a follow-up.

Also related: #2383 asks whether the DRA driver has folded into GPU Operator upstream. Scope item 4 here offers "EFA device plugin (or networking DRA driver, per #1326)", so #2383's answer bears on which arrangement this should wire up.