Document which machine types each accelerator criterion actually supports

[yuanchen8911]

Problem

--accelerator h100 names a GPU model, but a provider offers several machine
types per model, and the machine type — not the GPU — determines the fabric,
the sidecar, the NIC count, and the NCCL environment.

The docs do not say which machine type a recipe targets. A user on
a3-highgpu-8g reasonably expects this to work:

aicr recipe --accelerator h100 --service gke --os cos --intent training

It resolves to GPUDirect-TCPXO artifacts that cannot schedule on their nodes.
docs/user/cli-reference.md documents --accelerator as "Accelerator/GPU
type: h100, h200, gb200, ..." with no machine-type qualification. The only
place the targeting is stated is docs/integrator/gke-tcpxo-networking.md
("on a3-megagpu-8g nodes") — an integrator page a user picking criteria will
not read.

Scope of this issue

Documentation only. Document the supported machine type per accelerator ×
service, and state plainly what is unsupported.

Minimum targets:

• docs/user/cli-reference.md — qualify the --accelerator value list, or
  link it to a support matrix.
• Add a support matrix. Verified for GKE H100 against Google's GPUDirect
  documentation:

Machine type	Node label	Fabric	AICR support
a3-megagpu-8g	nvidia-h100-mega-80gb	GPUDirect-TCPXO	Supported
a3-highgpu-8g	nvidia-h100-80gb	GPUDirect-TCPX	Not supported (#2290)
a3-edgegpu-8g	nvidia-h100-80gb	GPUDirect-TCPX	Not supported (#2290)
a3-highgpu-1g/2g/4g	nvidia-h100-80gb	none — GPUDirect unsupported	Not supported

• Audit whether EKS and AKS have the same shape. Document them, or record
  that they were checked and do not.

Terminology note

The docs already use "SKU" to mean the GPU model — cli-reference.md says
"maps the device ID to a SKU, e.g. h100" and "accelerator model/SKU". New
text should say machine type (the provider's own term) so the same word
does not carry two meanings on the same page.

Open questions (design — not resolved by this issue)

1. Should AICR support multiple machine types per accelerator at all, or stay
   single-machine-type-per-recipe and document the boundary?
2. Fine-grained accelerator values (h100-mega-80gb vs h100) — reuses
   the existing axis with no model change, but mixes vendor GPU names with
   provider label names, multiplies the enum across every surface that
   documents it (the OpenAPI contract, six doc pages, issue templates), and
   breaks --accelerator h100 as a working input.
3. A new criteria axis — either machine type (precise, but puts
   provider-specific vocabulary in a provider-neutral model) or fabric /
   transport class (tcpxo, tcpx, efa, ib, none — provider-neutral,
   and the axis nccl-all-reduce-bw training gate is a fixed absolute fabric-specific busbw value applied to SKU-agnostic recipes → false-fails EKS/H100 small SKUs #1256 concluded is the one that matters).
4. Named profile runtimes, no model change — a user referencing
   torch-distributed-tcpx has already distinguished the case themselves.

Constraint on option 2: a3-highgpu-8g and a3-highgpu-1g share the label
nvidia-h100-80gb but differ on whether GPUDirect exists at all. The
accelerator label therefore cannot express the full ternary split
(TCPXO / TCPX / none), which is a wall the other options do not hit.

Out of scope

• Implementing GPUDirect-TCPX support — GKE TCPX: SKU-agnostic H100 recipes select TCPXO artifacts they cannot use #2290 owns that.
• The RequiredGPUNICNetworks = 8 floor conflict — GKE TCPX: SKU-agnostic H100 recipes select TCPXO artifacts they cannot use #2290 owns that.

Related

• GKE TCPX: SKU-agnostic H100 recipes select TCPXO artifacts they cannot use #2290 — GKE TCPX: A3 High/Edge selects TCPXO artifacts it cannot use
• [Epic]: No fabric-wired runtime ships, so validation can only measure a fixture #2217 — parent epic

[yuanchen8911]

Addressed by #2380 (draft).

That PR adds a Qualified machine types table to the aicr recipe reference, separating the two failure modes: GKE H100 fails hard (GPUDirect-TCPXO DaemonSets pin node affinity, so nothing schedules and deployment validation fails), while the EKS/AKS/GB200 families deploy normally and only the full-node performance floors are affected. It also records the same assumption as a comment on each affected overlay's criteria block.

The EKS/AKS audit this issue asked for is done — the result is that hard/soft split. No component on those paths pins to an instance type; the one AKS nodeAffinity is a capability pin (pci-15b3.present) whose health check has no desiredNumberScheduled > 0 assert, so a non-IB shape still passes deployment validation.

The four design questions above are deliberately left open — that PR documents the current boundary and changes no criteria axis, enum, or behavior. This issue stays open for the design decision.

Related: #2290 (the concrete defect this documents), #2378.