Skip to content

Commit 164fdb1

Browse files
anurag4DSBanurag4DSB
committed
S3CSI-204: Add TLS configuration documentation
Add dedicated TLS configuration guide covering prerequisites, Helm values, verification steps, and troubleshooting. Update Helm chart reference with tls.* parameters. Add CA/PEM/TLS terms to glossary. Add TLS note to volume provisioning docs. Issue: S3CSI-204
1 parent 5ae289d commit 164fdb1

6 files changed

Lines changed: 245 additions & 0 deletions

File tree

‎docs/concepts-and-reference/helm-chart-configuration-reference.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,25 @@ value: {{ coalesce .Values.node.s3EndpointUrl .Values.s3.endpointUrl | quote }}
153153
| `mountpointPod.headroomImage.tag` | Image tag for headroom pods. | `3.10` | No |
154154
| `mountpointPod.headroomImage.pullPolicy` | Image pull policy for headroom pods. | `IfNotPresent` | No |
155155

156+
## TLS Configuration
157+
158+
<!-- markdownlint-disable MD046 -->
159+
!!! info "Custom CA Certificates"
160+
When your S3 endpoint uses TLS with a private or internal CA, configure the `tls.*` parameters to inject the CA certificate.
161+
The ConfigMap must exist in **two** namespaces (controller and mounter pod) because they run in separate namespaces.
162+
See the [TLS Configuration Guide](../driver-deployment/tls-configuration.md) for ordering constraints and setup instructions.
163+
<!-- markdownlint-enable MD046 -->
164+
165+
| Parameter | Description | Default | Required |
166+
|------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------|-----------------------------|
167+
| `tls.caCertConfigMap` | Name of the ConfigMap containing the CA certificate bundle (key: `ca-bundle.crt`). Must exist in both the controller namespace and `mountpointPod.namespace`. Create in the controller namespace **before** Helm install, and in `mountpointPod.namespace` **after** (since Helm creates that namespace). If missing from either namespace, the respective pod stays in `ContainerCreating`. Leave empty to disable. | `""` | No |
168+
| `tls.initImage.repository` | Image repository for the CA certificate installation initContainer in mounter pods. | `alpine` | No |
169+
| `tls.initImage.tag` | Image tag for the CA certificate installation initContainer. | `3.21` | No |
170+
| `tls.initImage.pullPolicy` | Pull policy for the CA certificate init image. | `IfNotPresent` | No |
171+
| `tls.initResources.requests.cpu` | CPU request for the CA certificate init container. | `10m` | No |
172+
| `tls.initResources.requests.memory` | Memory request for the CA certificate init container. | `16Mi` | No |
173+
| `tls.initResources.limits.memory` | Memory limit for the CA certificate init container. | `64Mi` | No |
174+
156175
## CRD Cleanup Configuration (v2.0)
157176

158177
| Parameter | Description | Default | Required |

‎docs/driver-deployment/tls-configuration.md‎

Lines changed: 215 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,215 @@
1+
# TLS Configuration
2+
3+
## Overview
4+
5+
When your S3 endpoint uses TLS with certificates signed by a private or internal CA,
6+
the CSI driver needs access to the CA certificate to validate the connection.
7+
The Scality CSI Driver supports injecting custom CA certificates via Kubernetes ConfigMaps.
8+
9+
This is required when:
10+
11+
- Your RING S3 endpoint uses HTTPS with a self-signed or internally-signed certificate
12+
- Your organization uses a private CA for internal services
13+
- The S3 endpoint's certificate chain is not in the default system trust store
14+
15+
## Prerequisites
16+
17+
- A PEM-encoded CA certificate file (the root or intermediate CA that signed your S3 server certificate)
18+
- The CSI driver Helm chart installed or ready to install
19+
20+
## Configuration
21+
22+
### Step 1: Create the CA Certificate ConfigMap in the Controller Namespace
23+
24+
Create a ConfigMap containing your CA certificate in the CSI driver namespace
25+
(typically `kube-system`):
26+
27+
```bash
28+
kubectl create configmap s3-ca-cert \
29+
--from-file=ca-bundle.crt=/path/to/your/ca.crt \
30+
-n kube-system
31+
```
32+
33+
<!-- markdownlint-disable MD046 -->
34+
!!! important "Key Name"
35+
The ConfigMap key **must** be `ca-bundle.crt`. This is the key the driver expects.
36+
37+
!!! info "Why Two Namespaces?"
38+
The CA certificate ConfigMap must exist in **two** namespaces because the controller and
39+
mounter pods run in separate namespaces:
40+
41+
1. **Controller namespace** (e.g., `kube-system`) — mounted by the `s3-csi-controller` for
42+
AWS SDK S3 API calls (bucket creation/deletion during dynamic provisioning).
43+
2. **Mounter pod namespace** (e.g., `mount-s3`) — mounted by mounter pod init containers
44+
that inject the CA into the `mount-s3` trust store.
45+
46+
The mounter pod namespace (`mount-s3`) is created by the Helm chart, so you cannot create
47+
the ConfigMap there until after the Helm install (see Step 3).
48+
49+
!!! warning "Namespace Ordering"
50+
Do **not** attempt to create the ConfigMap in the `mount-s3` namespace before the Helm install —
51+
the namespace does not exist yet. If a ConfigMap is missing from either namespace, the
52+
respective pod will be stuck in `ContainerCreating` with a `configmap not found` event.
53+
<!-- markdownlint-enable MD046 -->
54+
55+
### Step 2: Install or Upgrade the Helm Chart
56+
57+
Set the `tls.caCertConfigMap` value to the name of your ConfigMap:
58+
59+
```bash
60+
helm upgrade --install scality-s3-csi \
61+
./charts/scality-mountpoint-s3-csi-driver \
62+
--namespace kube-system \
63+
--set s3.endpointUrl=https://s3.example.com:443 \
64+
--set tls.caCertConfigMap=s3-ca-cert
65+
```
66+
67+
### Step 3: Create the CA Certificate ConfigMap in the Mounter Namespace
68+
69+
After Helm creates the `mount-s3` namespace, create the same ConfigMap there:
70+
71+
```bash
72+
kubectl create configmap s3-ca-cert \
73+
--from-file=ca-bundle.crt=/path/to/your/ca.crt \
74+
-n mount-s3
75+
```
76+
77+
Mounter pods are created on-demand when workloads mount S3 volumes, so this ConfigMap
78+
just needs to exist before any workload pod starts.
79+
80+
### Step 4: Verify
81+
82+
Check that the controller pod has the CA certificate mounted:
83+
84+
```bash
85+
kubectl exec -n kube-system deploy/s3-csi-controller \
86+
-c s3-csi-controller -- ls /etc/ssl/custom-ca/
87+
```
88+
89+
Expected output: `ca-bundle.crt`
90+
91+
Verify the ConfigMap exists in the mounter pod namespace:
92+
93+
```bash
94+
kubectl get configmap s3-ca-cert -n mount-s3
95+
```
96+
97+
## How It Works
98+
99+
The TLS configuration operates at two levels:
100+
101+
### Controller Pod (Dynamic Provisioning)
102+
103+
The controller pod uses the CA certificate for S3 API calls (bucket creation/deletion)
104+
during dynamic provisioning:
105+
106+
- The ConfigMap is mounted at `/etc/ssl/custom-ca/` in the `s3-csi-controller` container
107+
- The `AWS_CA_BUNDLE` environment variable is set to `/etc/ssl/custom-ca/ca-bundle.crt`
108+
- AWS SDK Go v2 reads this variable and uses the CA certificate for TLS validation
109+
110+
### Mounter Pods (Volume Mounting)
111+
112+
Mounter pods use `mount-s3` (which uses s2n-tls) to mount S3 buckets.
113+
s2n-tls reads CA certificates from the system trust store (`/etc/ssl/certs/`),
114+
so a simple volume mount is not sufficient. Instead:
115+
116+
1. An **initContainer** (`install-ca-cert`) runs before the main `mountpoint` container
117+
2. The initContainer copies the system CA bundle from the Alpine image to a shared emptyDir volume
118+
3. It appends the custom CA certificate from the ConfigMap to the combined bundle
119+
4. The main container mounts the shared volume at `/etc/ssl/certs/` (read-only)
120+
5. `mount-s3` reads the combined trust store and validates the S3 endpoint certificate
121+
122+
The initContainer runs as non-root and complies with the PodSecurity `restricted` policy
123+
enforced on the mounter pod namespace.
124+
125+
## Helm Values Reference
126+
127+
| Parameter | Description | Default |
128+
| --------- | ----------- | ------- |
129+
| `tls.caCertConfigMap` | Name of the ConfigMap containing the CA certificate | `""` (disabled) |
130+
| `tls.initImage.repository` | Image repository for the CA cert init container | `alpine` |
131+
| `tls.initImage.tag` | Image tag for the CA cert init container | `3.21` |
132+
| `tls.initImage.pullPolicy` | Pull policy for the init image | `IfNotPresent` |
133+
| `tls.initResources.requests.cpu` | CPU request for the init container | `10m` |
134+
| `tls.initResources.requests.memory` | Memory request for the init container | `16Mi` |
135+
| `tls.initResources.limits.memory` | Memory limit for the init container | `64Mi` |
136+
137+
## Why ConfigMap Instead of Secret
138+
139+
CA certificates are public configuration data, not confidential information.
140+
Using ConfigMaps instead of Secrets:
141+
142+
- Follows the Kubernetes convention of using ConfigMaps for non-sensitive configuration
143+
- Avoids unnecessary RBAC complexity for managing Secrets
144+
- Makes the certificates easier to inspect and manage
145+
146+
## Troubleshooting
147+
148+
### Pod Stuck in ContainerCreating
149+
150+
If a controller or mounter pod is stuck in `ContainerCreating` after enabling TLS, the CA
151+
certificate ConfigMap is likely missing from that pod's namespace. Check the pod events:
152+
153+
```bash
154+
kubectl describe pod <pod-name> -n <namespace>
155+
```
156+
157+
Look for an event like: `configmap "s3-ca-cert" not found`.
158+
159+
To fix, create the ConfigMap in the correct namespace:
160+
161+
```bash
162+
# For controller pods stuck in ContainerCreating
163+
kubectl create configmap s3-ca-cert \
164+
--from-file=ca-bundle.crt=/path/to/your/ca.crt \
165+
-n kube-system
166+
167+
# For mounter pods stuck in ContainerCreating
168+
kubectl create configmap s3-ca-cert \
169+
--from-file=ca-bundle.crt=/path/to/your/ca.crt \
170+
-n mount-s3
171+
```
172+
173+
### Certificate Not Found
174+
175+
If mounter pods fail with TLS errors, verify the ConfigMap exists in **both** namespaces:
176+
177+
1. Controller namespace:
178+
179+
```bash
180+
kubectl get configmap s3-ca-cert -n kube-system
181+
```
182+
183+
2. Mounter pod namespace:
184+
185+
```bash
186+
kubectl get configmap s3-ca-cert -n mount-s3
187+
```
188+
189+
3. The ConfigMap has the correct key:
190+
191+
```bash
192+
kubectl get configmap s3-ca-cert -n mount-s3 -o jsonpath='{.data}' | head -c 100
193+
```
194+
195+
### Certificate Chain Issues
196+
197+
If you see certificate verification errors despite having the CA cert configured:
198+
199+
- Ensure you are providing the **root CA** certificate, not the server certificate
200+
- If using an intermediate CA, include the full chain in the `ca-bundle.crt` file
201+
- Verify the certificate is in PEM format (starts with `-----BEGIN CERTIFICATE-----`)
202+
203+
### Init Container Failures
204+
205+
If the init container fails, check its logs:
206+
207+
```bash
208+
kubectl logs <mounter-pod-name> -n mount-s3 -c install-ca-cert
209+
```
210+
211+
Common issues:
212+
213+
- The init image must include a system CA bundle at `/etc/ssl/certs/ca-certificates.crt`
214+
(Alpine includes this by default via the `ca-certificates` package)
215+
- The ConfigMap may not be mounted correctly

‎docs/glossary.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,29 +7,36 @@ This glossary defines acronyms, technical terms, and concepts used throughout th
77
| Acronym | Full Form | Definition |
88
|---------|-----------|------------|
99
| **API** | Application Programming Interface | A set of protocols and tools for building software applications |
10+
| **CA** | Certificate Authority | Trusted entity that issues digital certificates for verifying identity |
1011
| **CLI** | Command Line Interface | A text-based interface for interacting with software |
1112
| **CRD** | Custom Resource Definition | Kubernetes extension mechanism for defining custom resources |
1213
| **CRT** | Common Runtime | AWS Common Runtime library used for S3 operations |
1314
| **CSI** | Container Storage Interface | A standard for exposing storage systems to containerized workloads |
1415
| **DNS** | Domain Name System | System that translates domain names to IP addresses |
16+
| **FUSE** | Filesystem in Userspace | Mechanism allowing non-privileged users to create file systems without editing kernel code |
1517
| **GID** | Group Identifier | Numeric identifier for a group in Unix-like systems |
1618
| **GHCR** | GitHub Container Registry | GitHub's container image registry service |
1719
| **HTTP** | Hypertext Transfer Protocol | Protocol for transferring data over the web |
1820
| **HTTPS** | HTTP Secure | Secure version of HTTP using encryption |
1921
| **IAM** | Identity and Access Management | System for managing user identities and permissions |
2022
| **JSON** | JavaScript Object Notation | Lightweight data interchange format |
2123
| **KMS** | Key Management Service | Service for managing encryption keys |
24+
| **PEM** | Privacy-Enhanced Mail | Text encoding format for cryptographic keys and certificates |
2225
| **POSIX** | Portable Operating System Interface | Set of standards for Unix-like operating systems |
2326
| **PV** | PersistentVolume | Kubernetes resource representing a piece of storage |
2427
| **PVC** | PersistentVolumeClaim | Kubernetes resource requesting storage from a PV |
2528
| **RBAC** | Role-Based Access Control | Method of restricting access based on user roles |
2629
| **S3** | Simple Storage Service | Object storage service protocol |
2730
| **S3PA** | S3 Pod Attachment (MountpointS3PodAttachment) | Kubernetes custom resource that tracks which workload pods are attached to a specific S3 volume, enabling volume sharing and mounter pod lifecycle management |
31+
| **s2n-tls** | Signal to Noise TLS | AWS's open-source TLS implementation used by mount-s3 |
2832
| **SDK** | Software Development Kit | Collection of tools for developing applications |
33+
| **SSL** | Secure Sockets Layer | Predecessor to TLS, often used colloquially to mean TLS |
2934
| **SSE** | Server-Side Encryption | Encryption of data at rest on the server |
35+
| **TLS** | Transport Layer Security | Cryptographic protocol for secure communication over networks |
3036
| **TTL** | Time To Live | Duration for which data is considered valid |
3137
| **UID** | User Identifier | Numeric identifier for a user in Unix-like systems |
3238
| **URL** | Uniform Resource Locator | Web address identifying a resource |
39+
| **X.509** | X.509 | ITU-T standard for public key certificates, used in TLS |
3340
| **YAML** | YAML Ain't Markup Language | Human-readable data serialization standard |
3441

3542
## Technical Terms

‎docs/troubleshooting.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ aws s3 ls s3://your-bucket --endpoint-url https://your-s3-endpoint.com
3636
| Pod cannot write/delete files | Missing write permissions | Add `allow-delete` and/or `allow-overwrite` to PV `mountOptions` |
3737
| `MountVolume.SetUp failed: context deadline exceeded` with mounter pod log showing `accept unix /comm/mount.sock: i/o timeout` | Mounter pod missing FSGroup in security context | Upgrade to the latest release. As a workaround, remove `fsGroup` from workload pod's security context |
3838
| Pod stuck in `ContainerCreating` with "driver name s3.csi.scality.com not found in the list of registered CSI drivers" | CSI driver not yet registered (startup race condition) | Apply `s3.csi.scality.com/agent-not-ready:NoExecute` taint to nodes. See [Node Startup Taint](driver-deployment/node-startup-taint.md) |
39+
| Pod stuck in `ContainerCreating` with `configmap "..." not found` event | CA certificate ConfigMap missing from the pod's namespace | Create the ConfigMap in the correct namespace. See [TLS Troubleshooting](driver-deployment/tls-configuration.md#pod-stuck-in-containercreating) |
3940

4041
### Mount Issues
4142

@@ -46,6 +47,7 @@ aws s3 ls s3://your-bucket --endpoint-url https://your-s3-endpoint.com
4647
| "Access Denied" | Invalid S3 credentials | 1. Check secret contains `access_key_id` and `secret_access_key`<br/>2. Test credentials with AWS CLI<br/>3. Check bucket policy |
4748
| "InvalidBucketName" | Bucket name issue | 1. Check bucket exists<br/>2. Check bucket name format<br/>3. Ensure no typos |
4849
| "AWS_ENDPOINT_URL environment variable must be set" | Missing endpoint configuration | Set `s3EndpointUrl` in Helm values or driver configuration |
50+
| TLS handshake failure or certificate verify failed | CA certificate ConfigMap missing or incorrect | Check ConfigMap exists in both `kube-system` and `mount-s3` namespaces with key `ca-bundle.crt`. See [TLS Configuration](driver-deployment/tls-configuration.md#certificate-not-found) |
4951

5052
### Volume Issues
5153

‎docs/volume-provisioning/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ The Scality CSI Driver for S3 supports two methods for creating and managing per
3131
**Common Configuration:**
3232

3333
- [Mount Options Reference](mount-options.md) - Customization options for both provisioning methods
34+
- [TLS Configuration](../driver-deployment/tls-configuration.md) - Custom CA certificate support for HTTPS S3 endpoints
3435
<!-- markdownlint-enable MD046 -->
3536

3637
### Quick Start: Static Provisioning

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,7 @@ nav:
107107
- Installation Guide: driver-deployment/installation-guide.md
108108
- Upgrade Guide: driver-deployment/upgrade-guide.md
109109
- Node Startup Taint: driver-deployment/node-startup-taint.md
110+
- TLS Configuration: driver-deployment/tls-configuration.md
110111
- Uninstallation: driver-deployment/uninstallation.md
111112
- Volume Provisioning:
112113
- Overview: volume-provisioning/index.md

0 commit comments

Comments
 (0)