Skip to content

Commit 6d7ed35

Browse files
feat(recipes): functional kueue with default quota CRs
Kueue was registered in the component registry but referenced by zero overlays and shipped no queue or quota CRs, so a deployed Kueue admitted nothing out of the box. - Add registry-level manifestFiles defaults to ComponentConfig: a component can now ship default manifests wherever a recipe references it; ref-declared (overlay/mixin) lists take precedence. No opt-out: an empty ref-declared list is indistinguishable from absent — declare a replacement list to suppress the defaults. - Fail closed at registry load when a Kustomize-typed component declares manifestFiles: validateComponentRef rejects the combination per-ref, so a registry-level mistake would otherwise surface at resolution time in every consumer at once. - Mark kueue manifestsUseChartCRDs: the quota CRs instantiate CRDs the chart itself installs, so the injected kueue-post wrapper needs the helm-diff REST-mapper bypass (#914/#929 pattern, added in #1687). - Bump kueue chart 0.17.1 -> 0.18.2 and ship default quota CRs (ResourceFlavor with GPU-taint toleration, ClusterQueue with admission-neutral quotas, LocalQueue default/default) as post-install hooks ordered by dependency. - Pin managerConfig to the chart 0.18.2 default with integrations.frameworks trimmed to batch/job, JobSet, TrainJob - drops the default-on pod/deployment/statefulset webhook interception surface. - Health check now asserts the quota CRs reconcile to Active (assert-only, read-only allowlist compliant); documented coupling to the default CR names for overlays overriding manifestFiles. - Docs: registry field reference, catalog entry incl. upgrade/uninstall caveats and health-check coupling, BOM regen. Review fixes (PR #1686 non-blocking review): mark manifestsUseChartCRDs on kueue per #1687 coordination; fail closed at registry load when a Kustomize-typed component declares manifestFiles; assert the load-time guard returns ErrCodeInvalidRequest in its test; health-check, catalog, and godoc coupling notes for manifestFiles overrides; single-default-flavor toleration trade-off note. Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>
1 parent 471a57c commit 6d7ed35

13 files changed

Lines changed: 454 additions & 8 deletions

File tree

docs/contributor/component.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,10 @@ operator:
8383

8484
- `validations:` — bundle-time misconfiguration warnings ([validator.md](validator.md#component-validations-bundle-time))
8585
- `healthCheck.assertFile:` — chainsaw conformance assertions ([validator.md](validator.md))
86+
- `manifestFiles:` — manifest YAMLs (paths relative to the recipes data
87+
root) bundled with the component whenever a recipe references it and the
88+
componentRef declares none; shipped in the injected `-post` local chart
89+
after the main release. Ref-declared lists take precedence.
8690
- `storageClassPaths:` — where `--storage-class` is injected
8791
- `podScheduling.workload.workloadSelectorPaths` — for workload-pod placement
8892
- `gkeCriticalPriority`, `hasSelfRefCRDs`, `manifestsUseChartCRDs` — narrow service-specific quirks (see godoc on `ComponentConfig` for when these apply)
@@ -137,6 +141,7 @@ One-liner per field:
137141
| `storageClassPaths` | Where `--storage-class` is written |
138142
| `validations` | Bundle-time component check list ([validator.md](validator.md#component-validations-bundle-time)) |
139143
| `healthCheck.assertFile` | Chainsaw assert YAML path (relative to data dir) |
144+
| `manifestFiles` | Default manifest YAML paths bundled when the componentRef declares none (ref-declared lists take precedence). No opt-out: an empty ref-declared list is indistinguishable from absent (len == 0 → defaults filled) — to suppress the defaults, declare a replacement list. Helm components only; the loader rejects the combination with `kustomize:` |
140145
| `gkeCriticalPriority`, `hasSelfRefCRDs`, `manifestsUseChartCRDs` | Narrow service-specific flags (see godoc) |
141146

142147
## `nodeScheduling.system` vs `accelerated`

docs/user/component-catalog.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ The source of truth is [`recipes/registry.yaml`](https://github.com/NVIDIA/aicr/
3434
| **agentgateway-crds** | Custom Resource Definitions for agentgateway (Kubernetes Gateway API implementation for AI/ML inference). | [agentgateway](https://github.com/agentgateway/agentgateway) |
3535
| **agentgateway** | Kubernetes Gateway API implementation for AI/ML inference. Implements the Gateway API Inference Extension for model-aware ingress routing to InferencePool backends. | [agentgateway](https://github.com/agentgateway/agentgateway) |
3636
| **k8s-nim-operator** | NVIDIA NIM Operator for managing NIM (NVIDIA Inference Microservices) deployments on Kubernetes. | [K8s NIM Operator](https://github.com/NVIDIA/k8s-nim-operator) |
37-
| **kueue** | Kubernetes-native job queuing system. Manages quotas and admits jobs for batch and AI workloads. | [Kueue](https://github.com/kubernetes-sigs/kueue) |
37+
| **kueue** | Kubernetes-native job queuing system. Manages quotas and admits jobs for batch and AI workloads. Ships default quota CRs (ResourceFlavor `default-flavor`, ClusterQueue `cluster-queue`, LocalQueue `default` in the `default` namespace) so admission works out of the box — tune the ClusterQueue's nominal quotas to cluster capacity to enact real limits. Managed frameworks are pinned to batch/job, JobSet, and TrainJob. Upgrade note: the quota CRs are helm post-install/post-upgrade hooks with a delete-and-recreate policy — quiesce queues before upgrading the bundle (Kueue's resource-in-use finalizer on an active ClusterQueue/ResourceFlavor blocks the delete and can wedge the upgrade), and re-apply tuned quotas afterwards since upgrades reset them to the shipped defaults. Uninstalling leaves the hook-created CRs behind; delete them manually when removing Kueue. Overlays that override the component's `manifestFiles` (replacing the default quota CRs) must also override its health check — the shipped check asserts the default CR names above. | [Kueue](https://github.com/kubernetes-sigs/kueue) |
3838
| **kubeflow-trainer** | Kubeflow Training Operator for distributed training jobs (PyTorch, etc.). Manages multi-node training job lifecycle with JobSet integration. | [Kubeflow Trainer](https://github.com/kubeflow/trainer) |
3939
| **slinky-slurm-operator-crds** | Custom Resource Definitions for the SchedMD Slinky Slurm operator. Installs the `slinky.slurm.net` CRDs (Controller, NodeSet, LoginSet, Accounting, RestApi, Token). Installed separately to support CRD lifecycle management. | [Slinky Slurm Operator](https://github.com/SlinkyProject/slurm-operator) |
4040
| **slinky-slurm-operator** | SchedMD Slinky Slurm operator and admission webhook. Manages the lifecycle of Slurm clusters declared via Slinky CRs (Controller, NodeSet, LoginSet, Accounting, RestApi, Token). AICR's system node-selector and toleration bundle flags apply to both deployments; affinity remains available through component values or typed overrides. | [Slinky Slurm Operator](https://github.com/SlinkyProject/slurm-operator) |

docs/user/container-images.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ _Rendering fidelity:_ `catalog-parity: charts are rendered with the shared recip
4848
| kai-scheduler | helm | kai-scheduler | v0.14.1 | 2 |
4949
| kube-prometheus-stack | helm | prometheus-community/kube-prometheus-stack | 84.4.0 | 8 |
5050
| kubeflow-trainer | helm | kubeflow-trainer | 2.2.0 | 3 |
51-
| kueue | helm | kueue | 0.17.1 | 1 |
51+
| kueue | helm | kueue | 0.18.2 | 1 |
5252
| network-operator | helm | nvidia/network-operator | 26.1.1 | 5 |
5353
| network-operator-ocp | manifest ||| 0 |
5454
| network-operator-ocp-olm | manifest ||| 0 |
@@ -184,7 +184,7 @@ _No images extracted._
184184

185185
### kueue
186186

187-
- `registry.k8s.io/kueue/kueue:v0.17.1`
187+
- `registry.k8s.io/kueue/kueue:v0.18.2`
188188

189189
### network-operator
190190

pkg/recipe/components.go

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,18 @@ type ComponentConfig struct {
7070
// HealthCheck defines custom health check configuration for this component.
7171
HealthCheck HealthCheckConfig `yaml:"healthCheck,omitempty"`
7272

73+
// ManifestFiles lists manifest files (relative to the recipes data
74+
// root, e.g. "components/kueue/manifests/cluster-queue.yaml") bundled
75+
// with the component whenever a recipe references it and the
76+
// componentRef does not declare its own manifestFiles. A ref-declared
77+
// (overlay/mixin) manifestFiles list takes precedence — these defaults
78+
// are not merged into it. Consumers cannot opt out of the defaults
79+
// except by declaring their own list: an empty ref-declared list is
80+
// indistinguishable from an absent one after YAML decoding (len == 0
81+
// → defaults filled), so `manifestFiles: []` silently receives the
82+
// defaults back.
83+
ManifestFiles []string `yaml:"manifestFiles,omitempty"`
84+
7385
// GKECriticalPriority signals that the component's default chart manifests
7486
// include pods with `priorityClassName: system-node-critical` or
7587
// `system-cluster-critical`. When true and the recipe's
@@ -394,6 +406,15 @@ func loadComponentRegistryFor(provider DataProvider) (*ComponentRegistry, error)
394406
return nil, errors.New(errors.ErrCodeInvalidRequest,
395407
fmt.Sprintf("registry component %q declares the reserved override key %q; it is reserved for --set deployer:<key> Argo deployer options", comp.Name, ReservedDeployerKey))
396408
}
409+
// Fail closed on Kustomize components declaring manifestFiles
410+
// defaults: validateComponentRef rejects Kustomize refs that carry
411+
// manifestFiles, so ApplyRegistryDefaults would clone-fill this
412+
// list into every referencing recipe and fail resolution in every
413+
// consumer at once. Surface the config mistake here, at load time.
414+
if comp.GetType() == ComponentTypeKustomize && len(comp.ManifestFiles) > 0 {
415+
return nil, errors.New(errors.ErrCodeInvalidRequest,
416+
fmt.Sprintf("registry component %q is Kustomize but declares manifestFiles; a component may declare either Kustomize (tag/path) or raw manifest files, not both — manifestFiles defaults apply only to Helm components", comp.Name))
417+
}
397418
}
398419

399420
// Build index for fast lookup

pkg/recipe/components_test.go

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,15 @@
1515
package recipe
1616

1717
import (
18+
"bytes"
1819
"context"
20+
stderrors "errors"
1921
"maps"
2022
"slices"
2123
"strings"
2224
"testing"
2325

26+
"github.com/NVIDIA/aicr/pkg/errors"
2427
"gopkg.in/yaml.v3"
2528
)
2629

@@ -1200,3 +1203,85 @@ func TestLoadRegistry_RejectsReservedDeployerKey(t *testing.T) {
12001203
})
12011204
}
12021205
}
1206+
1207+
// TestLoadRegistry_RejectsKustomizeManifestFiles verifies the registry
1208+
// loader fails closed when a Kustomize-typed component declares
1209+
// registry-level manifestFiles defaults. validateComponentRef rejects
1210+
// Kustomize refs carrying manifestFiles, so a registry entry combining
1211+
// both would be clone-filled into every referencing recipe and fail
1212+
// resolution in every consumer at once — surface the config mistake at
1213+
// load time instead (same fail-closed contract as the reserved
1214+
// deployer-key guard above).
1215+
func TestLoadRegistry_RejectsKustomizeManifestFiles(t *testing.T) {
1216+
registryYAML := "apiVersion: aicr.run/v1alpha2\n" +
1217+
"kind: ComponentRegistry\n" +
1218+
"components:\n" +
1219+
" - name: my-kustomize-app\n" +
1220+
" displayName: My Kustomize App\n" +
1221+
" kustomize:\n" +
1222+
" defaultSource: https://github.com/example/my-app\n" +
1223+
" defaultPath: deploy/production\n" +
1224+
" defaultTag: v1.0.0\n" +
1225+
" manifestFiles:\n" +
1226+
" - components/my-kustomize-app/manifests/extra.yaml\n"
1227+
dp := newInMemoryProvider("kustomize-manifestfiles", map[string][]byte{
1228+
"registry.yaml": []byte(registryYAML),
1229+
})
1230+
_, err := GetComponentRegistryFor(dp)
1231+
if err == nil {
1232+
t.Fatal("expected registry load to fail on Kustomize component with manifestFiles, got nil error")
1233+
}
1234+
if !strings.Contains(err.Error(), `"my-kustomize-app"`) {
1235+
t.Errorf("error %q does not name the offending component", err.Error())
1236+
}
1237+
if !strings.Contains(err.Error(), "manifestFiles") {
1238+
t.Errorf("error %q does not mention manifestFiles", err.Error())
1239+
}
1240+
// Fail-closed contract: the guard must surface a 4xx invalid-request
1241+
// (a registry misconfiguration is a bad input, not an internal fault),
1242+
// mirroring the coherence-check precedent in
1243+
// componentref_coherence_test.go. Asserting only the message would stay
1244+
// green if the guard's code silently regressed to ErrCodeInternal.
1245+
if !stderrors.Is(err, errors.New(errors.ErrCodeInvalidRequest, "")) {
1246+
t.Errorf("want ErrCodeInvalidRequest, got %v", err)
1247+
}
1248+
}
1249+
1250+
func TestComponentRegistry_ManifestFilesResolve(t *testing.T) {
1251+
provider := NewEmbeddedDataProvider(GetEmbeddedFS(), "")
1252+
registry, err := GetComponentRegistryFor(provider)
1253+
if err != nil {
1254+
t.Fatalf("failed to load component registry: %v", err)
1255+
}
1256+
for _, comp := range registry.Components {
1257+
for _, mf := range comp.ManifestFiles {
1258+
data, err := provider.ReadFile(context.Background(), mf)
1259+
if err != nil {
1260+
t.Errorf("component %q manifestFiles entry %q is not readable from embedded data: %v",
1261+
comp.Name, mf, err)
1262+
continue
1263+
}
1264+
if len(bytes.TrimSpace(data)) == 0 {
1265+
t.Errorf("component %q manifestFiles entry %q is empty", comp.Name, mf)
1266+
}
1267+
}
1268+
}
1269+
1270+
// Regression sentinel pinned to kueue: a generic "some component has
1271+
// manifestFiles" check stays green if kueue's quota entries are
1272+
// dropped while another component still declares a list. Assert the
1273+
// kueue component and its exact quota CR paths.
1274+
kueue := registry.Get("kueue")
1275+
if kueue == nil {
1276+
t.Fatal("registry has no kueue component")
1277+
}
1278+
wantManifests := []string{
1279+
"components/kueue/manifests/resource-flavor.yaml",
1280+
"components/kueue/manifests/cluster-queue.yaml",
1281+
"components/kueue/manifests/local-queue.yaml",
1282+
}
1283+
if !slices.Equal(kueue.ManifestFiles, wantManifests) {
1284+
t.Errorf("kueue manifestFiles = %v, want %v (dependency-ordered quota CRs)",
1285+
kueue.ManifestFiles, wantManifests)
1286+
}
1287+
}

pkg/recipe/metadata.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import (
1919
"encoding/json"
2020
"fmt"
2121
"log/slog"
22+
"slices"
2223
"sort"
2324
"strings"
2425

@@ -212,6 +213,14 @@ func (ref *ComponentRef) ApplyRegistryDefaults(config *ComponentConfig) {
212213
}
213214
}
214215

216+
// ManifestFiles: default from the registry only when the resolved ref
217+
// carries none. Clone (not alias) so per-ref mutations cannot leak
218+
// into the shared registry config (see slice-aliasing overlay-merge
219+
// anti-pattern in CLAUDE.md).
220+
if len(ref.ManifestFiles) == 0 && len(config.ManifestFiles) > 0 {
221+
ref.ManifestFiles = slices.Clone(config.ManifestFiles)
222+
}
223+
215224
// healthCheck.assertFile hydration is NOT performed in this method.
216225
// ApplyRegistryDefaults runs per-ref against a registry config and has no
217226
// DataProvider in scope, but assertFile content lives on disk (or in an

pkg/recipe/metadata_test.go

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ import (
3737
"context"
3838
"fmt"
3939
"reflect"
40+
"slices"
4041
"strings"
4142
"testing"
4243
)
@@ -2086,6 +2087,73 @@ func TestComponentRefApplyRegistryDefaults_HealthCheckAsserts(t *testing.T) {
20862087
})
20872088
}
20882089

2090+
func TestComponentRefApplyRegistryDefaults_ManifestFiles(t *testing.T) {
2091+
registryDefaults := []string{
2092+
"components/kueue/manifests/resource-flavor.yaml",
2093+
"components/kueue/manifests/cluster-queue.yaml",
2094+
}
2095+
2096+
t.Run("filled from registry defaults when ref has none", func(t *testing.T) {
2097+
config := &ComponentConfig{
2098+
Name: "test-helm",
2099+
DisplayName: "Test Helm",
2100+
ManifestFiles: registryDefaults,
2101+
Helm: HelmConfig{
2102+
DefaultRepository: "https://charts.example.com",
2103+
},
2104+
}
2105+
ref := &ComponentRef{Name: "test-helm"}
2106+
2107+
ref.ApplyRegistryDefaults(config)
2108+
2109+
if !slices.Equal(ref.ManifestFiles, registryDefaults) {
2110+
t.Errorf("ManifestFiles = %v, want %v", ref.ManifestFiles, registryDefaults)
2111+
}
2112+
// Must be a copy, not an alias: mutating the ref must not leak
2113+
// into the shared registry config.
2114+
ref.ManifestFiles[0] = "mutated"
2115+
if config.ManifestFiles[0] != "components/kueue/manifests/resource-flavor.yaml" {
2116+
t.Errorf("registry config aliased: config.ManifestFiles[0] = %q", config.ManifestFiles[0])
2117+
}
2118+
})
2119+
2120+
t.Run("ref-declared manifest files win", func(t *testing.T) {
2121+
config := &ComponentConfig{
2122+
Name: "test-helm",
2123+
DisplayName: "Test Helm",
2124+
ManifestFiles: registryDefaults,
2125+
Helm: HelmConfig{
2126+
DefaultRepository: "https://charts.example.com",
2127+
},
2128+
}
2129+
own := []string{"components/other/manifests/custom.yaml"}
2130+
ref := &ComponentRef{Name: "test-helm", ManifestFiles: own}
2131+
2132+
ref.ApplyRegistryDefaults(config)
2133+
2134+
if !slices.Equal(ref.ManifestFiles, own) {
2135+
t.Errorf("ManifestFiles = %v, want %v (registry defaults must not overwrite)", ref.ManifestFiles, own)
2136+
}
2137+
})
2138+
2139+
t.Run("no-op when registry declares none", func(t *testing.T) {
2140+
config := &ComponentConfig{
2141+
Name: "test-helm",
2142+
DisplayName: "Test Helm",
2143+
Helm: HelmConfig{
2144+
DefaultRepository: "https://charts.example.com",
2145+
},
2146+
}
2147+
ref := &ComponentRef{Name: "test-helm"}
2148+
2149+
ref.ApplyRegistryDefaults(config)
2150+
2151+
if len(ref.ManifestFiles) != 0 {
2152+
t.Errorf("ManifestFiles = %v, want empty", ref.ManifestFiles)
2153+
}
2154+
})
2155+
}
2156+
20892157
// TestComponentRefMergeWithPath verifies that the Path field is correctly merged
20902158
// when merging ComponentRefs (overlay into base).
20912159
func TestComponentRefMergeWithPath(t *testing.T) {

recipes/checks/kueue/health-check.yaml

Lines changed: 43 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,12 @@
1414

1515
# Kueue Health Check
1616
#
17-
# Validates that Kueue is running and healthy in the kueue-system namespace.
18-
# Checks that the kueue-controller-manager deployment has at least one
19-
# available replica and that no pods in the namespace are stuck in Pending,
20-
# Failed, or Unknown phases.
17+
# Validates that Kueue is running and healthy in the kueue-system
18+
# namespace, and that the default quota CRs shipped with the component
19+
# (ResourceFlavor default-flavor, ClusterQueue cluster-queue, LocalQueue
20+
# default/default) were admitted and reconciled to Active — a functional
21+
# proof (CRDs installed, controller reconciling, webhook admitting,
22+
# flavor resolving) within the assert/error-only allowlist.
2123
apiVersion: chainsaw.kyverno.io/v1alpha1
2224
kind: Test
2325
metadata:
@@ -96,3 +98,40 @@ spec:
9698
namespace: kueue-system
9799
status:
98100
((initContainerStatuses || `[]`)[?state.waiting.reason == 'CrashLoopBackOff' || state.waiting.reason == 'ImagePullBackOff' || state.waiting.reason == 'ErrImagePull' || state.waiting.reason == 'CreateContainerConfigError'] | length(@) > `0`): true
101+
- name: validate-quota-crs-active
102+
try:
103+
# The default quota CRs ship with the component via registry
104+
# manifestFiles (wave N+1), so they must exist wherever this
105+
# check runs. v1beta2 is the storage version in chart 0.18.2.
106+
#
107+
# COUPLING: these asserts are pinned to the DEFAULT CR names. An
108+
# overlay that overrides the component's manifestFiles (shipping
109+
# its own queue CRs under different names) must also override
110+
# healthCheck.assertFile, or this check fails on the missing
111+
# default names even though Kueue itself is healthy.
112+
- assert:
113+
resource:
114+
apiVersion: kueue.x-k8s.io/v1beta2
115+
kind: ResourceFlavor
116+
metadata:
117+
name: default-flavor
118+
# JMESPath projection over conditions for the same sliceNode
119+
# exact-length reason documented above: a raw conditions list
120+
# only matches when it is the SOLE condition present.
121+
- assert:
122+
resource:
123+
apiVersion: kueue.x-k8s.io/v1beta2
124+
kind: ClusterQueue
125+
metadata:
126+
name: cluster-queue
127+
status:
128+
((conditions || `[]`)[?type == 'Active' && status == 'True'] | length(@) > `0`): true
129+
- assert:
130+
resource:
131+
apiVersion: kueue.x-k8s.io/v1beta2
132+
kind: LocalQueue
133+
metadata:
134+
name: default
135+
namespace: default
136+
status:
137+
((conditions || `[]`)[?type == 'Active' && status == 'True'] | length(@) > `0`): true
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Default ClusterQueue for the AICR kueue component.
16+
# nominalQuota values are deliberately generous (admission-neutral): they
17+
# exceed any realistic cluster so Kueue admits everything until an
18+
# operator tunes them down to actual capacity to enact real quotas.
19+
# Workloads requesting resources NOT covered below (e.g. hugepages, rdma
20+
# devices) are inadmissible — extend coveredResources when such workloads
21+
# join the queue.
22+
apiVersion: kueue.x-k8s.io/v1beta2
23+
kind: ClusterQueue
24+
metadata:
25+
name: cluster-queue
26+
annotations:
27+
# Post-install hook required for custom apiVersions (see resource-flavor
28+
# .yaml). Weight 5: applied after default-flavor (weight 1), which this
29+
# ClusterQueue's resourceGroups.flavors reference by name.
30+
"helm.sh/hook": post-install,post-upgrade
31+
"helm.sh/hook-weight": "5"
32+
"helm.sh/hook-delete-policy": before-hook-creation
33+
spec:
34+
namespaceSelector: {} # admit workloads from every namespace
35+
resourceGroups:
36+
- coveredResources: ["cpu", "memory", "nvidia.com/gpu"]
37+
flavors:
38+
- name: default-flavor
39+
resources:
40+
- name: cpu
41+
nominalQuota: "100000"
42+
- name: memory
43+
nominalQuota: 1000Ti
44+
- name: nvidia.com/gpu
45+
nominalQuota: "100000"

0 commit comments

Comments
 (0)