Feature: three-level permissions (Cluster Admin → Tenant → Subtenant)

[yindia]

Summary

Support a three-level permission hierarchy in Capsule:

Cluster Admin  →  Tenant Owner  →  Subtenant Owner

Today Capsule has two effective tiers: cluster administrators and tenant owners. There is no way for a tenant owner to delegate a bounded portion of their tenant to a sub-team without involving a cluster admin.

Problem / Use case

Larger tenants (e.g. a business unit) need to hand a subset of their tenant to sub-teams (squads, projects, environments) so those sub-teams can self-serve without getting full tenant-owner power and without cluster-admin intervention.

There is currently no first-class concept for this nested, self-service delegation.

What I want

A Subtenant tier owned by a Tenant Owner, where:

• A tenant owner can create/manage subtenants under their own tenant (no cluster admin needed).
• A subtenant owner gets only a bounded slice of the parent tenant — a subset of namespaces, resource/namespace quota, and RBAC.
• A subtenant's authority is always a subset of the parent's — it can never exceed or escalate above the parent tenant.
• Deleting/limiting the parent tenant cascades correctly to its subtenants.

Out of scope for this issue

Implementation/design details (API shape, controllers, webhooks) — this issue is only to capture the feature requirement. Happy to discuss the approach separately.

[oliverbaehler]

If a subtenant just correlates to a subset of namespaces within a tenant, managed by the Tenant Owner, this is already working. If you can outline your use-case a bit further i can assist you with a concrete design and point you to the necessary resources.

If this is about a Tenant CR can have nesting (eg. a parent Tenant=, then that's currently not possible and we would have to evaluate if that makes sense.

[yindia]

@oliverbaehler Thanks for the quick reply, and for offering to help with the design. Let me lay out the real topology, because it narrows the question a fair bit.

Platform team runs several clusters. In each cluster they create N tenants. A tenant is pinned to its own nodepool and a fixed resource envelope. Between tenants we need hard isolation: no node sharing, and no cross-tenant traffic (enforced with Istio on top of the namespace boundary). That part Capsule already does well, and it stays the platform team's job.

The mental model we want is: a tenant is a cluster, a subtenant is a namespace. Inside a tenant, soft isolation is enough.

The thing that breaks namespace-as-a-service for us is CRDs. Tenants need their own operators and CRs, but CRDs are cluster-scoped, so handing tenants the ability to install them means version and schema conflicts between teams. The usual escape hatch is vcluster, which gives every team a control plane to operate and makes their life worse, not better. What we actually want is for the platform team to whitelist a set of CRDs per tenant, and for the tenant admin team to serve those to their subtenants without ever touching cluster scope.

So the responsibility split is:

• Platform team: clusters, tenants, nodepool pinning, cross-tenant network policy, the total envelope per tenant, and which CRDs a tenant is allowed to use.
• Tenant admin team: namespaces inside their tenant, who gets access to each one, how the tenant's budget is divided across them, and which of the whitelisted CRDs each subtenant can use.

Given a subtenant is just a namespace, you're right that most of the RBAC side already works. The gaps are:

1. Quota delegation. ResourcePool is cluster-scoped and administered by cluster admins. A tenant admin team can't carve their own tenant's pool into per-namespace slices, so every "give squad-a 4 more cores until Friday" comes back to the platform team. Claims are self-service, but there's no per-namespace ceiling underneath the tenant that the tenant admin controls.
2. CRD scoping. No way to say "tenant A may use these CRDs, and its tenant admin decides which subtenants get which". Today it's all-or-nothing at cluster scope, which is what pushes people to vcluster.
3. No object for the slice. Nothing says "this namespace is squad-a's, owned by these subjects, capped at this much, allowed these APIs". The boundary lives in naming conventions and our own automation, so drift and cleanup are ours to handle.
4. Unconditional inheritance. Anything the tenant admin creates has to carry the tenant's hard boundary: nodeSelector and tolerations for the tenant's nodepool, the Istio and network policy set, allowed registries. That has to be non-negotiable from inside the tenant.

So the question is narrower than the original issue: can a tenant owner subdivide their own tenant's resources and API surface across their namespaces, without cluster-admin involvement? Nested Tenant CRs aren't required for our case. Tenant-scoped quota delegation, a per-tenant CRD allowlist, and enforced inheritance of the parent's node and network constraints would cover it.

If part of this is already reachable with ResourcePool and the newer quota work, point me at the resources and I'll test them against this setup and report back.

[oliverbaehler]

Okay, not all requirementns are going to be covered as per today. Before i start i want to hint towards the new rulesystem: https://projectcapsule.dev/docs/rules/enforcement/

While this is currently deployed via Tenant CR it ends up in a RuleStatus CR, the idea in the future is that there can be aggregates of RuleStatuses. Meaning Rules can also be created per namespace by whomever has the permission to do so. But this will take some more thinking from my side until a implementation lands (months).

One other thing, we are also adding a Break-The-Glass Functionality, essentially where users can provide Templates, which require approvement and then something is kept for a time-period, this may also work well with your timed requirement:

# This template grants temporary CRUD access to all instances of one custom
# resource in the namespace of the BreakRequest.
#
# Example BreakRequest parameters:
# params:
#   name: widgets-editor-alice
#   crdName: widgets.platform.example.io
#   subjectKind: User
#   subjectName: alice@example.com
apiVersion: capsule.clastix.io/v1beta2
kind: BreakRequestTemplate
metadata:
  name: crd-resource-crud
spec:
  # Optional. Selectors are ORed; omit this field to allow BreakRequests from
  # every namespace. Matching namespace names are published in status.namespaces.
  # namespaceSelectors:
  #   - matchLabels:
  #       projectcapsule.dev/break-the-glass: enabled
  autoApprove: true
  defaultDuration: 1h
  maxDuration: 8h
  keepFor: 7d
  paramSchema:
    type: object
    additionalProperties: false
    required:
      - name
      - crdName
      - subjectKind
      - subjectName
    properties:
      name:
        type: string
        description: Unique DNS-compatible name for the temporary Role and RoleBinding.
        pattern: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?$'
        maxLength: 63
      crdName:
        type: string
        description: Name of the CustomResourceDefinition, for example widgets.platform.example.io.
        minLength: 1
        maxLength: 253
      subjectKind:
        type: string
        description: Kind of RBAC subject receiving temporary access.
        enum:
          - User
          - Group
      subjectName:
        type: string
        description: Name of the Kubernetes user or group receiving temporary access.
        minLength: 1
  context:
    resources:
      - apiVersion: apiextensions.k8s.io/v1
        kind: CustomResourceDefinition
        name: '{{ .crdName }}'
        index: crd
        optional: false
  resources:
    - policy:
        # Owner requires Capsule to create the resource. Merge adopts an
        # existing resource when possible and creates it otherwise.
        creation: Owner
        # Prevent changes through admission while the request is active.
        protect: true
        force: false
      targets:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: Role
          metadata:
            name: '{{ .name }}'
          rules:
            - apiGroups:
                - '{{ (index .crd 0).spec.group }}'
              resources:
                - '{{ (index .crd 0).spec.names.plural }}'
              verbs:
                - get
                - list
                - watch
                - create
                - update
                - patch
                - delete
                - deletecollection
      template: |
        # Multi YAML
        ---
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          name: {{ $.params.name }}
        rules:
          - apiGroups:
              - {{ (index $.context.resources.crd 0).spec.group }}
            resources:
              - {{ (index $.context.resources.crd 0).spec.names.plural }}
            verbs:
              - get
              - list
              - watch
              - create
              - update
              - patch
              - delete
              - deletecollection

    - policy:
        creation: Owner
        protect: true
        force: false
      targets:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            name: '{{ .name }}'
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: Role
            name: '{{ .name }}'
          subjects:
            - apiGroup: rbac.authorization.k8s.io
              kind: '{{ .subjectKind }}'
              name: '{{ .subjectName }}'

I will try to solve these issues with tools available as per release 0.14.2:

1. Quota Delegation: Okay, but why can't they. In your setup you would probably have a Resourcepool per Tenant, which equals the capacity of the Node, since these are also dedicated to the Tenant. This is provisioned by the platform, we can also write a GTR to automate that process so that Resourcepools scale based on the nodes but that would be a bit more advanced.So if the TenantOwner can now distribute the Claims to the namespaces, don't we create a ceiling per namespace and the total potential quantity matches the available resources from the node?

2. CRD permissions: "tenant A may use these CRDs, and its tenant admin decides which subtenants get which"", indicates that it's just about being able to use CRDs, that's more a question of native RBAC than anything else. If we provide Tenant-Owners with the proper permissions to access CRDs, they can then also distribute further bindings within their Tenant with TenantResources. A simple example:

We want to allow Tenant-Owners to distribute access to Gateway API, for this we must first distribute these privileges to the Tenant-Owners, in a strict scenario the controller needs to be able to bind the role as well:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gateway-api-role-distributor
rules:
  - apiGroups:
      - rbac.authorization.k8s.io
    resources:
      - rolebindings
    verbs:
      - get
      - list
      - watch
      - create
      - update
      - patch
      - delete
  - apiGroups:
      - rbac.authorization.k8s.io
    resources:
      - clusterroles
    resourceNames:
      - gateway-api-role-distributor
    verbs:
      - get
      - bind
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: capsule:controller:gateway-api-role-distributor
  labels:
    projectcapsule.dev/aggregate-to-controller: "true"
rules:
  - apiGroups:
      - rbac.authorization.k8s.io
    resources:
      - clusterroles
    resourceNames:
      - gateway-api-role-distributor
    verbs:
      - get
      - bind

On the Tenant we will leverage Promotions

---
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: solar
  labels:
    customer: renewable
  annotations:
    info.projectcapsule.dev/icon: https://cdn-icons-png.flaticon.com/512/3463/3463440.png
    info.projectcapsule.dev/description: "Production tenant for the solar team"
    info.projectcapsule.dev/links: '[{"title":"Grafana","url":"https://grafana.example.com/d/payments","icon":"mdi:grafana"},{"title":"Runbook","url":"https://wiki.example.com/payments-runbook", "icon":  "icon": "fa-solid fa-chart-line"}]'
spec:
  rules:
    # Promote only the dedicated TenantResource ServiceAccount. Capsule binds
    # this distributor role in every solar namespace so the TenantResource can
    # create the temporary user-facing RoleBindings there.
    - permissions:
        promotions:
          - clusterRoles:
              - gateway-api-role-distributor
              - admin

Let's assume every Tenant has *-system namespace where the Tenant-Owners will be acting from, for "cluster orchestration". There the tenant-owner can now distribute this binding to the rest of the tenant or slices:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gateway-api-role-distributor
  namespace: solar-system
  labels:
    projectcapsule.dev/promote: "true"
---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: rbac-distribution
  namespace: solar-system
spec:
  resyncPeriod: 60s
  serviceAccount:
    name: gateway-api-role-distributor
  resources:
    - rawItems:
        - apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            name: "tenant:cr:gateway-api"
            namespace: "{{namespace}}"
          subjects:
          - kind: User
            name: dave
            apiGroup: rbac.authorization.k8s.io
          roleRef:
            kind: ClusterRole
            name: gateway-api-role-distributor
            apiGroup: rbac.authorization.k8s.io

3. That's correct, however in such a *-system namespace you could just have a configmap where the Tenant-Owner can manage this data, which can then be loaded in combination with eg. TenantResources. If we revisit the above example, we can do something like this:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: tenant-management
  namespace: solar-system
data:
  team-a: |
    owners:
      - kind: User
        name: alice
      - kind: User
        name: bob
  team-b: |
    owners:
      - kind: User
        name: alice
      - kind: User
        name: bob
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gateway-api-role-distributor
  namespace: solar-system
  labels:
    projectcapsule.dev/promote: "true"
---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-distribution
  namespace: solar-system
spec:
  resyncPeriod: 60s
  serviceAccount:
    name: gateway-api-role-distributor
  resources:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: solar-system
       context:
        resources:
          - index: mgmt
            apiVersion: v1
            kind: ConfigMap
            name: tenant-management
            namespace: solar-system
      generators:
        - template: |
            {{- range $team, $config := (index $.mgmt 0 "data") }}
            {{- $team_config := fromYAML $config }}
            ---
            apiVersion: capsule.clastix.io/v1beta2
            kind: TenantResource
            metadata:
              name: "{{ $team }}-tenant-distribution"
              namespace: solar-system
              ownerReferences:
                - apiVersion: capsule.clastix.io/v1beta2
                  kind: TenantResource
                  name: {{ $.replications.metadata.name | quote }}
                  uid: {{ $.replications.metadata.uid | quote }}
                  controller: true
                  blockOwnerDeletion: true
            spec:
              resyncPeriod: 60s
              serviceAccount:
                name: gateway-api-role-distributor
              resources:
                - namespaceSelector:
                    matchLabels:
                      team: {{ $team }}
                  rawItems:
                    - apiVersion: rbac.authorization.k8s.io/v1
                      kind: RoleBinding
                      metadata:
                        name: "tenant:{{`{{namespace}}`}}:cr:gateway-api"
                        namespace: "{{`{{namespace}}`}}"
                      subjects:
                      {{- range $team_config.owners }}
                      - kind: {{ .kind }}
                        name: {{ .name }}
                        apiGroup: rbac.authorization.k8s.io
                      {{- end }}
                      roleRef:
                        kind: ClusterRole
                        name: gateway-api-role-distributor
                        apiGroup: rbac.authorization.k8s.io
            {{- end }}

We have an example inventory in a ConfigMap, based on that we spawn for each "team" a dedicated TenantResource, which then distributes the required objects into the sub slices. I would recommend here a dedicated CRD in your case just to have some api validation already. With this technique you can achieve what you are looking for - i found a bug with this case, will be fixed in 0.14.3.

4. This is true, currently this requires the platform involvement. For namespaced resources you can use the above approach. For the other requirements it might be easiest to use another policy framework to mutate the tolerations etc. We currently don't have that. But we are already working on having better scheduling enforcement + RuleStatus being available to TenantOwners. So that requirement can be currently not fully satisfied.

Hope that helps, we are obiously happy to add features, if we think they might be a benefit for your case and the project. Maybe in the end we can add a reference documentation for this case on the website, WDYT?

(,,>﹏<,,)

[yindia]

@oliverbaehler This helps a lot, thanks. Model I'm confirming: tenant = cluster, subtenant = namespace, soft isolation inside.

1. Quota. ResourcePool per tenant + owner handing out claims gets me most of the way. The bit I'm stuck on: claims are self-service per namespace, so nothing stops one namespace draining the whole pool. orderedQueue sorts ordering, not a cap. I want the tenant admin to say "squad-a max 4 cores, squad-b max 8" without calling cluster-admin. Does a per-namespace ceiling the owner controls exist today, or is it "grant claim RBAC and review sizes"? Fine if it's soft, just want to know.

2. CRDs. Makes sense, it's RBAC + Promotions + TenantResource. The gateway-api-role-distributor role with bind/aggregate-to-controller, promoted then fanned out, is exactly the split I wanted. I'll test it on Gateway API and a couple of my own operators.

3. Slice object. The *-system ConfigMap → generator TenantResource works, and I'll take the dedicated-CRD suggestion for the validation. That's the object I was faking with naming conventions. Will wait for the 0.14.3 fix before testing.

4. Inheritance. Understood, nodeSelector/tolerations/netpol need an external policy engine for now. The scheduling enforcement + RuleStatus for owners is what would close it natively, so I'll watch that.

So 2 and 3 I can validate now, 1 is my one open question, 4 is roadmap.

[oliverbaehler]

1. So essentially that's what i suggested. Let's extend that gtr example:

---
apiVersion: capsule.clastix.io/v1beta2
kind: ResourcePool
metadata:
  name: solar
  labels:
    projectcapsule.dev/tenant: solar
spec:
  quota:
    hard:
      limits.cpu: "0"
      requests.cpu: "2"
      requests.memory: "2Gi"
      limits.memory: "4Gi"
      requests.nvidia.com/gpu: 8
  selectors:
  - matchLabels:
      capsule.clastix.io/tenant: solar
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: tenant-management
  namespace: solar-system
data:
  team-a: |
    quota:
      requests.cpu: "1"
      requests.memory: "1Gi"
      limits.memory: "2Gi"
      requests.nvidia.com/gpu: 4
    owners:
      - kind: User
        name: alice
      - kind: User
        name: bob
  team-b: |
    quota:
      requests.cpu: "1"
      requests.memory: "1Gi"
      limits.memory: "2Gi"
      requests.nvidia.com/gpu: 4
    owners:
      - kind: User
        name: alice
      - kind: User
        name: bob
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: gateway-api-role-distributor
  namespace: solar-system
  labels:
    projectcapsule.dev/promote: "true"
---
apiVersion: capsule.clastix.io/v1beta2
kind: TenantResource
metadata:
  name: tenant-distribution
  namespace: solar-system
spec:
  resyncPeriod: 60s
  serviceAccount:
    name: gateway-api-role-distributor
  resources:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: solar-system
      context:
        resources:
          - index: mgmt
            apiVersion: v1
            kind: ConfigMap
            name: tenant-management
            namespace: solar-system
      generators:
        - template: |
            {{- range $team, $config := (index $.mgmt 0 "data") }}
            {{- $team_config := fromYAML $config }}
            ---
            apiVersion: capsule.clastix.io/v1beta2
            kind: TenantResource
            metadata:
              name: "{{ $team }}-tenant-distribution"
              namespace: solar-system
              ownerReferences:
                - apiVersion: capsule.clastix.io/v1beta2
                  kind: TenantResource
                  name: {{ $.replications.metadata.name | quote }}
                  uid: {{ $.replications.metadata.uid | quote }}
                  controller: true
                  blockOwnerDeletion: true
            spec:
              resyncPeriod: 60s
              serviceAccount:
                name: gateway-api-role-distributor
              resources:
                - namespaceSelector:
                    matchLabels:
                      team: {{ $team }}
                  generators:
                    - template: |
                        ---
                        apiVersion: capsule.clastix.io/v1beta2
                        kind: ResourcePoolClaim
                        metadata:
                          name: "managed-quota"
                          namespace: "{{`{{ $.namespace.metadata.name }}`}}"
                        spec:
                          pool: "solar"
                          claim:
                            {{- $team_config.quota | toYAML | nindent 16 }}
                  rawItems:
                    - apiVersion: rbac.authorization.k8s.io/v1
                      kind: RoleBinding
                      metadata:
                        name: "tenant:{{`{{namespace}}`}}:cr:gateway-api"
                        namespace: "{{`{{namespace}}`}}"
                      subjects:
                      {{- range $team_config.owners }}
                      - kind: {{ .kind }}
                        name: {{ .name }}
                        apiGroup: rbac.authorization.k8s.io
                      {{- end }}
                      roleRef:
                        kind: ClusterRole
                        name: gateway-api-role-distributor
                        apiGroup: rbac.authorization.k8s.io
            {{- end }}