RestateCluster: pod lifecycle, sidecars, termination grace period, and extra volumes#153
Merged
Merged
Conversation
…teCluster Add three optional fields to RestateCluster `spec.compute`, passed through to the StatefulSet pod template: - `lifecycle`: postStart / preStop hooks for the Restate container. - `sidecars`: native sidecar containers (appended to initContainers with restartPolicy forced to Always so they start before and terminate after the Restate container, rather than blocking startup as plain init containers). - `terminationGracePeriodSeconds`: overrides the previously hardcoded 60s. All three reuse the k8s_openapi types and are optional; defaults are unchanged. The Pkl bindings map `lifecycle`/`sidecars` to the pkl-k8s Lifecycle/Container types via converters so they reference the imports instead of expanding inline.
…teCluster Add two optional fields to RestateCluster `spec.compute`, passed through to the StatefulSet pod template: - `extraVolumes`: appended to the pod's volumes (alongside operator-managed ones). - `extraVolumeMounts`: appended to the Restate container's volume mounts. This lets lifecycle hooks and sidecars reference reusable mounted content (e.g. a ConfigMap of hook scripts at /node-state-control) instead of only inline shell. Sidecars already carry their own volumeMounts and share these pod-level volumes. Both reuse the k8s_openapi Volume/VolumeMount types and are optional; the Pkl bindings map them to the pkl-k8s Volume/VolumeMount types via converters.
pcholakov
force-pushed
the
pod-lifecycle-passthrough
branch
from
June 30, 2026 14:27
917f101 to
2886e6c
Compare
…, doc caveats - Fix the extraVolumes reserved-name docs: the real operator-managed volumes are "combined-ca-certs"/"trusted-ca-<n>" (trustedCaCerts) and "request-signing-private-key-secret"/"-provider" (requestSigningPrivateKey), not "trustedCaCerts"/"requestSigningPrivateKey". The previous text would have led users to avoid safe names and collide with the real ones. - Validate volume-name uniqueness in the reconciler (validate_unique_volume_names), returning a clear InvalidRestateConfig error instead of relying on the API server's opaque "Duplicate value" rejection. Covers extra-vs-operator and extra-vs-extra collisions. - Document that postStart runs concurrently with the entrypoint and is not a reliable pre-start hook (use command/args for strict ordering). - Note native sidecars require Kubernetes 1.29+ (GA 1.33). - Note terminationGracePeriodSeconds: 0 means immediate SIGKILL (skips preStop). - Add the five new spec.compute fields to the README schema table. - Release note: use ["/bin/sh", "<script>"] for the preStop example.
…tPaths) The previous validate_unique_volume_names only iterated template.spec.volumes, which never contains "storage" -- that name exists as a volumeClaimTemplate. So an extraVolume named "storage" passed validation despite the error message and docs listing it as reserved, and the StatefulSet controller would then silently drop the user's definition in favor of the PVC-backed volume. - Seed the seen-set from volumeClaimTemplate names so a "storage" collision errors like the others. - Also reject duplicate mountPaths on the Restate container (closes the extraVolumeMounts gap); duplicate paths -- not names -- are what the API server rejects, since a volume may legitimately be mounted at multiple paths. - Rename to validate_pod_volumes; extend the test to cover storage and mountPath.
Contributor
|
@pcholakov you mentioned this in our call. it's a draft so wanted to confirm if it's ready for review? |
pcholakov
marked this pull request as ready for review
July 1, 2026 11:10
Contributor
Author
|
@lukebond ready for review now, was just testing some experimental scripts and wanted to make sure there aren't any missing features/config knobs that I still needed to add. All good as-is! |
lukebond
approved these changes
Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exposes five additional optional pass-through fields to
RestateClusterspec.compute, surfaced on the underlying StatefulSet pod template. All are optional; defaults are unchanged, so existing clusters are unaffected.New fields
lifecycle—postStart/preStophooks for the Restate container. Note: Kubernetes runspostStartconcurrently with the entrypoint, so it is not a reliable "before Restate starts" hook — usecommand/args(an entrypoint wrapper) for strict pre-start ordering.sidecars— native sidecar containers (appended toinitContainerswithrestartPolicy: Alwaysforced, so they start before and terminate after the Restate container). Requires native sidecar support (Kubernetes 1.29+, GA 1.33).terminationGracePeriodSeconds— overrides the previously hardcoded 60s.0means immediate SIGKILL (skipspreStop).extraVolumes— additional pod volumes alongside the operator-managed ones.extraVolumeMounts— additional mounts for the Restate container; sidecars carry their own mounts and share these pod-level volumes.Together these enable graceful-shutdown hooks, companion containers, and mounting reusable content (e.g. a ConfigMap of hook scripts) rather than only inline shell.
Volume-name collisions
extraVolumesnames must not collide with operator-managed volumes:storage,tmp,config(always),combined-ca-certsandtrusted-ca-<n>(whentrustedCaCertsis set), andrequest-signing-private-key-secret/request-signing-private-key-secret-provider(whenrequestSigningPrivateKeyis set). The operator validates this and rejects a collision with a clearInvalidRestateConfigerror rather than letting the API server reject the StatefulSet with an opaque message.Notes
k8s_openapitypes; the Pkl bindings map them to the pkl-k8sLifecycle/Container/Volume/VolumeMounttypes so the bindings reference imports rather than expanding inline.CRD YAML changes are purely additive (no existing field touched). Tests pass;
just generate-allreproducible; examples evaluate.