The mount strategy is the cheapest one: one pod mounts both PVCs and rsync copies locally, without SSH, a service or the pod network. It only works within one namespace, because a pod can only mount the PVCs of its own namespace.
Between two namespaces of the same cluster we fall back to clusterip. That one needs sshd, a key pair, a service, and the CNI has to allow the traffic. Where a network policy blocks it, the migration fails with nothing pointing at the policy.
Worth exploring whether a pod can reach both volumes across namespaces, at least in some setups, probably with elevated privileges. Ideas:
a. hostPath into the kubelet's volume directory. Run a holder pod in each namespace, pinned to the same node, so both volumes get mounted on that node. Then run the rsync pod on that node with hostPath mounts of <kubelet-root>/pods/<pod-uid>/volumes/<plugin>/<pv-name>/mount for both. Needs a pod that is allowed hostPath (Pod Security privileged level), the kubelet root directory (/var/lib/kubelet by default, but distributions differ, e.g., microk8s), and both volumes being mountable on one node at the same time. Works with any driver that mounts a filesystem, since the volume is then just a directory on the node.
b. A second PV pointing at the same backend. Copy the source PV spec into a new PV with a claimRef to a new PVC in the destination namespace, then mount both in one pod there. Only works when the driver tolerates the same volume handle being published twice, and with RWO volumes the attach/detach controller may refuse. Fragile, and it touches the user's PV objects, so risky.
c. CrossNamespaceVolumeDataSource (alpha feature gate, with a ReferenceGrant). That is cloning, not mounting, and it depends on the driver. Listing it to rule it out.
(a) looks like the only realistic one. Open questions:
- How much do we gain over
clusterip when both pods land on the same node anyway? The SSH traffic stays on the node in that case too. The gains would be: no dependency on the CNI and network policies, no sshd, no encryption overhead. Is that worth a privileged pod?
- Is the kubelet pod volume path stable enough across distributions (k3s, Talos, OpenShift, GKE, microk8s) to build on, or does it need a flag?
- Whatever comes out of it has to be opt-in and never in the default order, since it needs a privileged pod and a
hostPath mount.
The
mountstrategy is the cheapest one: one pod mounts both PVCs and rsync copies locally, without SSH, a service or the pod network. It only works within one namespace, because a pod can only mount the PVCs of its own namespace.Between two namespaces of the same cluster we fall back to
clusterip. That one needs sshd, a key pair, a service, and the CNI has to allow the traffic. Where a network policy blocks it, the migration fails with nothing pointing at the policy.Worth exploring whether a pod can reach both volumes across namespaces, at least in some setups, probably with elevated privileges. Ideas:
a.
hostPathinto the kubelet's volume directory. Run a holder pod in each namespace, pinned to the same node, so both volumes get mounted on that node. Then run the rsync pod on that node withhostPathmounts of<kubelet-root>/pods/<pod-uid>/volumes/<plugin>/<pv-name>/mountfor both. Needs a pod that is allowedhostPath(Pod Securityprivilegedlevel), the kubelet root directory (/var/lib/kubeletby default, but distributions differ, e.g., microk8s), and both volumes being mountable on one node at the same time. Works with any driver that mounts a filesystem, since the volume is then just a directory on the node.b. A second PV pointing at the same backend. Copy the source PV spec into a new PV with a
claimRefto a new PVC in the destination namespace, then mount both in one pod there. Only works when the driver tolerates the same volume handle being published twice, and with RWO volumes the attach/detach controller may refuse. Fragile, and it touches the user's PV objects, so risky.c.
CrossNamespaceVolumeDataSource(alpha feature gate, with aReferenceGrant). That is cloning, not mounting, and it depends on the driver. Listing it to rule it out.(a) looks like the only realistic one. Open questions:
clusteripwhen both pods land on the same node anyway? The SSH traffic stays on the node in that case too. The gains would be: no dependency on the CNI and network policies, no sshd, no encryption overhead. Is that worth a privileged pod?hostPathmount.