Guide to setting up local-path storage

[dancreed]

For the life of me (I'm a relative newb to Omni and kubernetes) I can't figure out in Omni how to write a patch or whatnot to get local-path storage working. Can someone smarter than I please help?

Thanks,
Dan
Dan.Creed@thecreeds.net

[rothgar]

By default omni will deploy talos and partition the disk with an EPHEMERAL partition. You can use that partition to mount host volumes or you can use something like the local-path-provisioner to create volumes for workloads dynamically.

https://docs.siderolabs.com/kubernetes-guides/csi/local-storage#local-path-provisioner

The one important thing you need to do is when you create the cluster you need to add the patches to make sure the disk is partitioned correctly during installation time.

[dancreed]

Thanks any step by step guide to doing that? The whole patching thing, in which because I am running omni (not just talos) is throwing me for a loop, as there is no workers.yaml

[rothgar]

You can define patches when you're creating the cluster and those patches can be applied to individual nodes, machine sets, or the entire cluster. You can also apply patches via cluster templates.

https://docs.siderolabs.com/omni/reference/cluster-templates#cluster-templates

[Unix4ever]

Thanks any step by step guide to doing that? The whole patching thing, in which because I am running omni (not just talos) is throwing me for a loop, as there is no workers.yaml

With Omni you don't manage the full machine config like you would do with Talos, so you don't have workers.yaml.

Step by step guide for creating the patches can be found here: https://docs.siderolabs.com/omni/omni-cluster-setup/create-a-patch-for-cluster-machines

You will need to create it for worker nodes.

By the way with the latest Talos you can use UserVolumes: https://docs.siderolabs.com/talos/v1.11/reference/configuration/block/uservolumeconfig#uservolumeconfig

The config patch example can be something like this:

apiVersion: v1alpha1
kind: UserVolumeConfig
name: local-path-volume
provisioning:
    diskSelector:
        match: disk.transport == "nvme" # The Common Expression Language (CEL) expression to match the disk.
        # we do not define the maxSize here to make it take all available space

Selectors might be a bit tricky to write. You can get available fields by doing:

talosctl get disks -n <your-node-uuid>

You can use disk-by-id to match it. It will be the most stable selector:

        diskSelector:
          match: '"/dev/disk/by-id/ata-TOSHIBA-TR150_46AB52****" in disk.symlinks'

The rest depends on your setup:

• if you use VMs you can just use a separate block device for the local path.
• if you have a bare metal machine with the large disk where Talos is installed you can reserve a part of this disk for Talos and give the rest for the UserVolume. This is done with VolumeConfig:

apiVersion: v1alpha1
kind: VolumeConfig
name: EPHEMERAL
provisioning:
  diskSelector:
     match: '"/dev/disk/by-id/ata-TOSHIBA-TR150_46AB52****" in disk.symlinks'
   minSize: 32GB
   maxSize: 64GB
---
apiVersion: v1alpha1
kind: UserVolumeConfig
name: local-path-volume
provisioning:
  diskSelector:
    match: '"/dev/disk/by-id/ata-TOSHIBA-TR150_46AB52****" in disk.symlinks'

When you add a volume like this it will be mounted as /var/mnt/local-path-volume on the nodes.