Skip to content

Commit cc8bef1

Browse files
winlinuxmattwinlinuxmatt
committed
Update README with Longhorn and system extensions documentation
- Added Talos System Extensions section documenting custom image with iscsi-tools, util-linux-tools, and qemu-guest-agent - Added comprehensive Persistent Storage with Longhorn section including: - Installation instructions with Helm - Pod security configuration steps - Verification commands - UI access information - Updated Key Features to highlight custom Talos image and Longhorn storage - Updated Directory Structure to include longhorn-values.yaml and LONGHORN_TALOS_SETUP.md - Added talosctl commands for verifying system extensions - Linked to detailed LONGHORN_TALOS_SETUP.md for troubleshooting
1 parent 9bf2281 commit cc8bef1

1 file changed

Lines changed: 118 additions & 6 deletions

File tree

‎README.md‎

Lines changed: 118 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,10 @@ This repository provides Infrastructure as Code (IaC) for deploying and managing
1717
### Key Features
1818

1919
- **Declarative VM Provisioning:** Proxmox VMs for control plane and worker nodes are managed via Terraform.
20+
- **Custom Talos Image:** Uses Talos Image Factory with baked-in system extensions (iSCSI tools, util-linux tools) for persistent storage support.
2021
- **Talos OS & Kubernetes Versioning:** Talos and Kubernetes versions are parameterized in `variables.tf` for easy upgrades.
2122
- **Automated Cluster Configuration:** Talos machine configurations are generated and applied automatically to each node.
23+
- **Longhorn Persistent Storage:** Distributed block storage with NodePort UI access (port 30080) for volume management.
2224
- **Rolling Upgrades:** Change a version variable and apply to safely upgrade Talos and/or Kubernetes across your cluster.
2325
- **CI/CD Linting:** A GitHub Actions workflow automatically checks Terraform formatting and lints code on pull requests and pushes to `main`.
2426

@@ -43,13 +45,18 @@ This repository provides Infrastructure as Code (IaC) for deploying and managing
4345
4446
```
4547
.
46-
├── cluster.tf # Talos cluster and machine configuration resources
47-
├── files.tf # Talos image download and local variables
48-
├── providers.tf # Terraform provider configuration
49-
├── variables.tf # All input variables, including versioning
50-
├── virtual_machines.tf# Proxmox VM definitions for control plane and workers
48+
├── cluster.tf # Talos cluster and machine configuration resources
49+
├── files.tf # Talos custom image download with system extensions
50+
├── providers.tf # Terraform provider configuration
51+
├── variables.tf # All input variables, including versioning
52+
├── virtual_machines.tf # Proxmox VM definitions for control plane and workers
53+
├── longhorn-values.yaml # Helm values for Longhorn persistent storage
54+
├── LONGHORN_TALOS_SETUP.md # Longhorn setup guide and troubleshooting
55+
├── templates/
56+
│ ├── worker-disks.yaml.tmpl # Worker disk configuration template
57+
│ └── cpnetwork.yaml.tmpl # Control plane network template (unused)
5158
├── .github/workflows/terraform-lint.yml # CI workflow for linting
52-
└── README.md # Project documentation
59+
└── README.md # Project documentation
5360
```
5461
5562
### Automation
@@ -62,6 +69,98 @@ This repository provides Infrastructure as Code (IaC) for deploying and managing
6269
6370
---
6471
72+
## Talos System Extensions
73+
74+
This cluster uses a custom Talos image built via the [Talos Image Factory](https://factory.talos.dev/) with the following system extensions baked in:
75+
76+
- **iscsi-tools** (v0.2.0): Provides iSCSI initiator support for persistent storage
77+
- **util-linux-tools** (2.41.1): Additional Linux utilities for storage management
78+
- **qemu-guest-agent** (10.0.2): Enhanced VM integration with Proxmox
79+
80+
**Schematic ID**: `e187c9b90f773cd8c84e5a3265c5554ee787b2fe67b508d9f955e90e7ae8c96c`
81+
82+
These extensions enable Longhorn to properly manage persistent volumes on Talos nodes. The `ext-iscsid` service runs automatically on all nodes.
83+
84+
### Verify System Extensions
85+
86+
Check that extensions are installed on all nodes:
87+
```bash
88+
talosctl get extensions --nodes 10.0.0.70,10.0.0.71,10.0.0.72,10.0.0.73,10.0.0.74,10.0.0.75
89+
```
90+
91+
Verify iSCSI service is running:
92+
```bash
93+
talosctl get services --nodes 10.0.0.73,10.0.0.74,10.0.0.75 | grep ext-iscsid
94+
```
95+
96+
---
97+
98+
## Persistent Storage with Longhorn
99+
100+
This cluster includes [Longhorn](https://longhorn.io/) for distributed block storage across worker nodes.
101+
102+
### Longhorn Features
103+
104+
- **Distributed Storage**: Replicated volumes across multiple nodes for high availability
105+
- **Dynamic Provisioning**: Automatic PersistentVolume creation via StorageClasses
106+
- **Web UI**: Accessible via NodePort on port 30080 (http://NODE_IP:30080)
107+
- **Backup & Restore**: Volume snapshots and backup capabilities
108+
- **Pod Security**: Configured with privileged permissions in `longhorn-system` namespace
109+
110+
### Install Longhorn
111+
112+
Longhorn is deployed using Helm with custom values:
113+
114+
```bash
115+
helm repo add longhorn https://charts.longhorn.io
116+
helm repo update
117+
helm install longhorn longhorn/longhorn \
118+
--namespace longhorn-system \
119+
--create-namespace \
120+
--values longhorn-values.yaml
121+
```
122+
123+
### Configure Pod Security for Longhorn
124+
125+
Longhorn requires privileged pod security. Apply labels to the namespace:
126+
127+
```bash
128+
kubectl label namespace longhorn-system \
129+
pod-security.kubernetes.io/enforce=privileged \
130+
pod-security.kubernetes.io/audit=privileged \
131+
pod-security.kubernetes.io/warn=privileged
132+
```
133+
134+
### Verify Longhorn Installation
135+
136+
Check that all Longhorn pods are running:
137+
```bash
138+
kubectl get pods -n longhorn-system
139+
```
140+
141+
Verify storage classes are created:
142+
```bash
143+
kubectl get storageclass
144+
```
145+
146+
Expected output:
147+
```
148+
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
149+
longhorn (default) driver.longhorn.io Delete Immediate true 5m
150+
longhorn-static driver.longhorn.io Delete Immediate true 5m
151+
```
152+
153+
### Access Longhorn UI
154+
155+
Access the Longhorn web interface at:
156+
```
157+
http://<any-node-ip>:30080
158+
```
159+
160+
For detailed setup instructions, troubleshooting, and disk management, see [LONGHORN_TALOS_SETUP.md](LONGHORN_TALOS_SETUP.md).
161+
162+
---
163+
65164
## Additional Steps
66165

67166
After setting up the cluster, you may find the following steps helpful.
@@ -119,6 +218,19 @@ talos-worker-03 Ready <none> 90s v1.32.0
119218
...
120219
```
121220

221+
- **Verify System Extensions**:
222+
```bash
223+
talosctl get extensions --nodes 10.0.0.70
224+
```
225+
Sample output:
226+
```
227+
NODE NAMESPACE TYPE ID VERSION NAME VERSION
228+
10.0.0.70 runtime ExtensionStatus 0 1 iscsi-tools v0.2.0
229+
10.0.0.70 runtime ExtensionStatus 1 1 util-linux-tools 2.41.1
230+
10.0.0.70 runtime ExtensionStatus 2 1 qemu-guest-agent 10.0.2
231+
10.0.0.70 runtime ExtensionStatus 3 1 schematic e187c9b90f773cd8c84e5a3265c5554ee787b2fe67b508d9f955e90e7ae8c96c
232+
```
233+
122234
- **Health Dashboard Example**:
123235
![Talosctl Dashboard](photos/talosctl_dashboard.png)
124236

0 commit comments

Comments
 (0)