Add feature to install llm-d infra using kustomize for teams preferring native Kubernetes tooling

README.md

@@ -27,7 +27,7 @@ This repository provides the Kubernetes infrastructure components, Helm charts,
 - Gateway API v1.4.0+ installed
 - Gateway controller (Istio, Kgateway, AgentGateway or GKE) deployed in your cluster
 
-### Install llm-d Infrastructure
+### Install llm-d Infrastructure with Helm
 
 ```bash
 # Add the Helm repository
@@ -40,6 +40,24 @@ helm install my-llm-d-infra llm-d-infra/llm-d-infra
 
 For detailed configuration options, see the [Helm chart documentation](charts/llm-d-infra/README.md).
 
+## Installation llm-d Infrastructure with Kustomize
+
+For kustomize-based deployments, see [kustomize/README.md](kustomize/README.md).
+
+### Quick Start
+
+```bash
+# Istio gateway
+kubectl apply -k https://github.com/llm-d-incubation/llm-d-infra//kustomize/overlays/istio
+
+# Kgateway
+kubectl apply -k https://github.com/llm-d-incubation/llm-d-infra//kustomize/overlays/kgateway
+
+# GKE
+kubectl apply -k https://github.com/llm-d-incubation/llm-d-infra//kustomize/overlays/gke
+
+See kustomize/ for examples and customization guide.
+
 ## Documentation and Guides
 
 **Note:** All quickstart guides and user documentation have moved to the main llm-d repository:

kustomize/MIGRATION.md

@@ -0,0 +1,128 @@
+This guide helps you migrate from Helm to Kustomize for llm-d-infra.
+
+  ## Benefits which Kustomize brings with it
+
+  - No Helm templating - plain Kubernetes YAML
+  - GitOps-native (ArgoCD, Flux)
+  - Modular components (mix and match features)
+  - Easier to understand and debug
+
+  ## Quick Migration
+
+  ### Before (Helm)
+  ```bash
+  helm install gateway llm-d-infra/llm-d-infra \
+    --set gateway.gatewayClassName=istio \
+    --set ingress.enabled=true
+  ```
+  ### After (Kustomize)
+  ```bash
+  kubectl apply -k kustomize/examples/istio-with-ingress
+  ```
+  ### Mapping
+  ``` bash
+  | Helm                                 | Kustomize                           |
+  |--------------------------------------|-------------------------------------|
+  | Chart                                | overlays/istio or overlays/kgateway |
+  | --set gateway.gatewayClassName=istio | overlays/istio/                     |
+  | --set ingress.enabled=true           | components: [ingress]               |
+  | Custom values file                   | Create custom kustomization.yaml    |
+  | helm upgrade                         | kubectl apply -k                    |
+  | helm uninstall                       | kubectl delete -k                   |
+  ```
+  ### Step-by-Step guide
+
+  1. Identify Your Current Helm Values
+
+  `helm get values gateway -o yaml > current-values.yaml`
+
+  2. Find Equivalent Kustomize Configuration
+
+  Check examples/ for matching configuration.
+
+  3. Customize if Needed
+
+  Create your own kustomization:
+  ``` bash
+  # my-deployment/kustomization.yaml
+  resources:
+    - github.com/llm-d-incubation/llm-d-infra//kustomize/overlays/istio
+
+  namespace: my-namespace
+  namePrefix: my-
+
+  patches:
+    # Your customizations here
+  ```
+  4. Deploy
+
+  ``` bash 
+  kubectl apply -k my-deployment/
+  ```
+
+  ### Common Scenarios
+
+  #### Scenario: Custom Resource Limits
+
+  ##### Helm:
+  ``` bash
+  gateway:
+    gatewayParameters:
+      resources:
+        limits:
+          cpu: "4"
+          memory: 2Gi
+  ```
+  ##### Kustomize:
+  Use examples/istio-with-ingress as template (has production resources).
+
+  #### Scenario: Multiple Environments
+
+  ##### Helm:
+  ``` bash
+  helm install dev-gateway ... -f dev-values.yaml
+  helm install prod-gateway ... -f prod-values.yaml
+  ```
+
+  ##### Kustomize:
+  ``` bash
+  deployments/
+  ├── dev/
+  │   └── kustomization.yaml  # references overlays/istio
+  └── prod/
+      └── kustomization.yaml  # references examples/istio-with-tls
+  ```
+
+  #### Scenario: GitOps (ArgoCD)
+
+  ##### Helm:
+  ``` bash
+  source:
+    chart: llm-d-infra
+    repoURL: https://llm-d-incubation.github.io/llm-d-infra/
+    helm:
+      values: |
+        gateway:
+          gatewayClassName: istio
+  ```
+
+  ##### Kustomize:
+  ``` bash
+  source:
+    repoURL: https://github.com/llm-d-incubation/llm-d-infra
+    path: kustomize/overlays/istio
+  ```
+  ### Rollback Plan
+
+  Keep Helm deployed during migration:
+
+  1. Deploy kustomize to different namespace
+  2. Test thoroughly
+  3. Switch traffic
+  4. Remove Helm deployment
+
+  Need Help?
+
+  - See README.md for full documentation
+  - Check examples/ for common patterns
+  - Ask in #sig-installation Slack channel

kustomize/README.md

@@ -0,0 +1,174 @@
+# llm-d-infra Kustomize
+
+  Kustomize-based deployment for llm-d gateway infrastructure.
+
+  ## Quick Start
+
+  Choose a gateway provider and deploy:
+
+  ```bash
+  # Istio (default)
+  kubectl apply -k kustomize/overlays/istio
+
+  # Kgateway (Envoy Gateway)
+  kubectl apply -k kustomize/overlays/kgateway
+
+  # GKE L7 Regional External
+  kubectl apply -k kustomize/overlays/gke
+
+  Directory Structure
+
+  kustomize/
+  ├── base/                  # Common Gateway resource
+  ├── overlays/              # Gateway provider variants
+  │   ├── istio/            # Istio gateway
+  │   ├── kgateway/         # Kgateway (Envoy Gateway)
+  │   └── gke/              # GKE L7 gateway
+  ├── components/            # Optional features
+  │   ├── ingress/          # External Ingress
+  │   └── tls/              # HTTPS/TLS support
+  └── examples/              # Production-ready configurations
+      ├── istio-minimal/
+      ├── istio-with-ingress/
+      ├── istio-with-tls/
+      ├── kgateway-production/
+      └── gke-minimal/
+
+  Usage Patterns
+
+  Basic Development Setup
+
+  kubectl apply -k kustomize/overlays/istio
+
+  Production with Ingress
+
+  kubectl apply -k kustomize/examples/istio-with-ingress
+
+  Production with TLS
+
+  # Create TLS secret first
+  kubectl create secret tls llm-d-tls-cert \
+    --cert=tls.crt --key=tls.key -n llm-d-prod
+
+  # Deploy
+  kubectl apply -k kustomize/examples/istio-with-tls
+
+  Customization Guide
+
+  Change Namespace
+
+  # your-deployment/kustomization.yaml
+  resources:
+    - ../kustomize/overlays/istio
+
+  namespace: my-namespace
+
+  Add Name Prefix
+
+  namePrefix: dev-
+
+  Add Labels
+
+  commonLabels:
+    environment: production
+    team: ml-platform
+
+  Change Service Type to LoadBalancer
+
+  For Istio:
+  patches:
+    - patch: |-
+        - op: replace
+          path: /data/service
+          value: |
+            spec:
+              type: LoadBalancer
+      target:
+        kind: ConfigMap
+        name: llm-d-gateway
+
+  For Kgateway:
+  patches:
+    - patch: |-
+        - op: replace
+          path: /spec/kube/service/type
+          value: LoadBalancer
+      target:
+        kind: GatewayParameters
+
+  Add Custom Listeners
+
+  patches:
+    - patch: |-
+        - op: add
+          path: /spec/listeners/-
+          value:
+            name: metrics
+            port: 9090
+            protocol: HTTP
+            allowedRoutes:
+              namespaces:
+                from: Same
+      target:
+        kind: Gateway
+
+  Increase Resource Limits
+
+  See examples/istio-with-ingress for pattern of replacing ConfigMap with production version.
+
+  Add Annotations
+
+  patches:
+    - patch: |-
+        - op: add
+          path: /metadata/annotations
+          value:
+            prometheus.io/scrape: "true"
+            prometheus.io/port: "9090"
+      target:
+        kind: Gateway
+
+  Components
+
+  Components are optional features you can mix and match:
+
+  # your-deployment/kustomization.yaml
+  resources:
+    - ../kustomize/overlays/istio
+
+  components:
+    - ../kustomize/components/ingress  # Add external Ingress
+    - ../kustomize/components/tls      # Add HTTPS listener
+
+   Examples Explained
+
+  | Example | Gateway Provider | Use Case | Features |
+  |---------|-----------------|----------|----------|
+  | `istio-minimal` | Istio | Development, basic setup | ConfigMap + Telemetry |
+  | `istio-with-ingress` | Istio | Production, external access | + Ingress + Production resources |
+  | `istio-with-tls` | Istio | Production, HTTPS | + TLS listener (port 443) |
+  | `kgateway-production` | Kgateway | Production, Envoy Gateway | GatewayParameters + Production resources |
+  | `gke-minimal` | GKE | GKE clusters | Minimal GKE setup |
+  | `gke-tpu` | GKE | GKE with TPU workloads | TPU labels + annotations |
+
+  Testing Your Configuration
+
+  # Preview what will be deployed
+  kubectl kustomize kustomize/examples/istio-with-tls
+
+  # Save output for inspection
+  kubectl kustomize kustomize/overlays/istio > /tmp/gateway.yaml
+
+  Troubleshooting
+
+  Gateway not getting IP address:
+  - Check gateway class is installed: kubectl get gatewayclass
+  - Check gateway status: kubectl describe gateway -n llm-d
+
+  ConfigMap not applied:
+  - Verify parametersRef points to correct ConfigMap
+  - Check ConfigMap exists: kubectl get configmap -n llm-d
+
+  TLS not working:
+  - Verify TLS secret exists: kubectl get secret llm-d-tls-cert -n llm-d-prod
+  - Check certificate is valid: kubectl get secret llm-d-tls-cert -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text

kustomize/base/gateway.yaml

@@ -0,0 +1,20 @@
+# kustomize/base/gateway.yaml
+  apiVersion: gateway.networking.k8s.io/v1
+  kind: Gateway
+  metadata:
+    name: llm-d-gateway
+    labels:
+      app.kubernetes.io/name: llm-d-infra
+      app.kubernetes.io/component: inference-gateway
+  spec:
+    # Default to istio
+    gatewayClassName: istio
+
+    # Default listeners - HTTP on port 80
+    listeners:
+      - name: http
+        port: 80
+        protocol: HTTP
+        allowedRoutes:
+          namespaces:
+            from: All

kustomize/base/kustomization.yaml

@@ -0,0 +1,15 @@
+# kustomize/base/kustomization.yaml
+  apiVersion: kustomize.config.k8s.io/v1beta1
+  kind: Kustomization
+
+  namespace: llm-d
+
+  commonLabels:
+    app.kubernetes.io/part-of: llm-d
+    app.kubernetes.io/managed-by: kustomize
+
+  resources:
+    - gateway.yaml
+
+  # Default name prefix(optional, users can override)
+  # namePrefix: ""

kustomize/components/ingress/ingress.yaml

@@ -0,0 +1,18 @@
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: llm-d-gateway
+  labels:
+    app.kubernetes.io/component: ingress
+spec:
+  ingressClassName: nginx
+  rules:
+    - http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: llm-d-gateway-istio
+                port:
+                  number: 80