Skip to content

Commit f74782b

Browse files
wax911wax911
committed
feat: introduce automated Docker Swarm stack generation and enforcement
introduces a new system to automatically generate and validate Docker Swarm stack files from their source definitions. The `tools/generate_stacks.py` script now consolidates per-service `docker-compose.yml` files and new `swarm.fragment.yml` files into aggregated `stacks/*.yml` artifacts. all `docker-compose.yml` files now include an `x-stack:` annotation to define their target stack. to enforce consistency, a CI job checks for drift between source files and generated stacks. additionally, a nightly workflow automatically regenerates drifted stacks and opens pull requests. the `stackctl.sh` script gains `generate` and `sync` commands, and automatically regenerates stacks before deployment with `up`. common Docker Compose keys incompatible with Swarm (e.g., `container_name`, `restart`, `build`) are removed, default logging is applied, and relative paths are rewritten for Swarm compatibility. also updates Docker image dependencies across various services.
1 parent 84c9548 commit f74782b

52 files changed

Lines changed: 1785 additions & 734 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ on:
44
pull_request:
55
types: [opened, synchronize, reopened]
66
push:
7-
branches: [ main ]
7+
branches: [ main, dev ]
88

99
jobs:
1010
lint:
@@ -42,4 +42,37 @@ jobs:
4242
scan-type: 'fs'
4343
severity: 'HIGH,CRITICAL'
4444
exit-code: '1'
45-
ignore-unfixed: true
45+
ignore-unfixed: true
46+
47+
stack-sync-check:
48+
needs: [lint]
49+
name: Stack drift check
50+
runs-on: ubuntu-latest
51+
steps:
52+
- name: Checkout
53+
uses: actions/checkout@v6
54+
- name: Set up Python
55+
uses: actions/setup-python@v5
56+
with:
57+
python-version: '3.x'
58+
- name: Install PyYAML
59+
run: pip install PyYAML
60+
- name: Check stacks are in sync with compose sources
61+
run: python3 tools/generate_stacks.py --output-dir /tmp/stacks-check
62+
- name: Diff generated vs committed stacks
63+
run: |
64+
drift=0
65+
for stack in infrastructure observability platform; do
66+
if ! diff -q "/tmp/stacks-check/${stack}.yml" "stacks/${stack}.yml" >/dev/null 2>&1; then
67+
echo "DRIFT: stacks/${stack}.yml is out of sync"
68+
diff "stacks/${stack}.yml" "/tmp/stacks-check/${stack}.yml" || true
69+
drift=1
70+
else
71+
echo "OK: stacks/${stack}.yml"
72+
fi
73+
done
74+
if [ "$drift" -eq 1 ]; then
75+
echo ""
76+
echo "Run './stackctl.sh generate' and commit the result."
77+
exit 1
78+
fi

‎.github/workflows/nightly-stack-sync.yml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: Nightly stack sync
2+
3+
on:
4+
schedule:
5+
# Run every night at 02:00 UTC
6+
- cron: '0 2 * * *'
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: write
11+
pull-requests: write
12+
13+
jobs:
14+
sync:
15+
name: Regenerate stacks and open PR if drifted
16+
runs-on: ubuntu-latest
17+
steps:
18+
- name: Checkout dev branch
19+
uses: actions/checkout@v6
20+
with:
21+
ref: dev
22+
fetch-depth: 0
23+
24+
- name: Set up Python
25+
uses: actions/setup-python@v5
26+
with:
27+
python-version: '3.x'
28+
29+
- name: Install PyYAML
30+
run: pip install PyYAML
31+
32+
- name: Regenerate stacks
33+
run: python3 tools/generate_stacks.py
34+
35+
- name: Check for drift
36+
id: drift
37+
run: |
38+
if git diff --quiet stacks/; then
39+
echo "drifted=false" >> "$GITHUB_OUTPUT"
40+
else
41+
echo "drifted=true" >> "$GITHUB_OUTPUT"
42+
git diff --stat stacks/
43+
fi
44+
45+
- name: Open or update PR for drifted stacks
46+
if: steps.drift.outputs.drifted == 'true'
47+
env:
48+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
49+
run: |
50+
branch="chore/nightly-stack-sync-$(date +%Y%m%d)"
51+
git config user.name "github-actions[bot]"
52+
git config user.email "github-actions[bot]@users.noreply.github.com"
53+
git checkout -b "$branch"
54+
git add stacks/
55+
git commit -m "chore(stacks): nightly regeneration from compose sources [skip ci]"
56+
git push origin "$branch"
57+
gh pr create \
58+
--base dev \
59+
--head "$branch" \
60+
--title "chore(stacks): nightly stack regeneration $(date +%Y-%m-%d)" \
61+
--body "Automated PR: stacks/ drifted from compose sources. Generated by the nightly-stack-sync workflow.
62+
63+
Run \`./stackctl.sh sync\` locally to reproduce the diff." \
64+
|| echo "PR already exists or could not be created"

‎anitrend/docker-compose.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
---
2+
x-stack: platform
23
services:
34
django-server:
45
image: ghcr.io/anitrend/anitrend:0.8.0
@@ -20,7 +21,7 @@ services:
2021
driver: local
2122
options:
2223
max-size: "2m"
23-
max-file: "3"
24+
max-file: 3
2425

2526
networks:
2627
default:

‎anitrend/swarm.fragment.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
services:
2+
django-server:
3+
deploy:
4+
mode: replicated
5+
replicas: 1
6+
placement:
7+
constraints:
8+
- node.role == manager
9+
resources:
10+
reservations:
11+
memory: 128M
12+
cpus: '0.10'
13+
limits:
14+
memory: 384M
15+
cpus: '0.50'

‎apisix/api-dashboard/docker-compose.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
---
2+
x-stack: infrastructure
23
services:
34
apisix_dashboard:
45
container_name: "apisix-dashobard"
@@ -21,7 +22,7 @@ services:
2122
logging:
2223
options:
2324
max-size: "10m"
24-
max-file: "3"
25+
max-file: 3
2526

2627

2728
networks:

‎apisix/api-dashboard/swarm.fragment.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
services:
2+
apisix_dashboard:
3+
deploy:
4+
mode: global
5+
placement:
6+
constraints:
7+
- node.role == manager
8+
resources:
9+
reservations:
10+
memory: 64M
11+
limits:
12+
memory: 256M

‎apisix/api-gateway/docker-compose.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
---
2+
x-stack: infrastructure
23
volumes:
34
cache:
45
name: apisix-cache
@@ -29,7 +30,7 @@ services:
2930
logging:
3031
options:
3132
max-size: "10m"
32-
max-file: "3"
33+
max-file: 3
3334

3435

3536
networks:

‎apisix/api-gateway/swarm.fragment.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
services:
2+
apisix_gateway:
3+
deploy:
4+
mode: global
5+
placement:
6+
constraints:
7+
- node.role == manager
8+
resources:
9+
reservations:
10+
memory: 128M
11+
cpus: '0.10'
12+
limits:
13+
memory: 384M
14+
cpus: '0.40'

‎apisix/etcd/docker-compose.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
---
2+
x-stack: infrastructure
23
volumes:
34
etcd-data:
45
name: 'etcd-data'
@@ -20,7 +21,7 @@ services:
2021
logging:
2122
options:
2223
max-size: "10m"
23-
max-file: "3"
24+
max-file: 3
2425

2526
networks:
2627
default:

‎apisix/etcd/swarm.fragment.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
services:
2+
etcd:
3+
deploy:
4+
resources:
5+
reservations:
6+
memory: 64M
7+
limits:
8+
memory: 256M

0 commit comments

Comments
 (0)