Skip to content

Commit 2fbd6a8

Browse files
julienldclaude
andauthored
fix: replace cron with systemd for demo server (prevents process leak) (#1110)
* fix(internal): replace cron with systemd for demo server to prevent process leaks Weekly cron was spawning new hamcp-test-env processes without killing old ones, accumulating 60+ zombie processes over 34 days (one new process per week per cron run). Systemd service ensures a single instance via ExecStart, auto-restarts on failure, and the weekly timer triggers a clean restart via ExecStartPost. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(internal): add sudoers rule and use sudo in update service for hamcp-demo restart Without the sudoers rule, the weekly update timer (running as SETUP_USER) cannot restart the systemd service. ExecStartPost also needs explicit sudo to cross the privilege boundary since it inherits the User= context from ExecStart. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(internal): guard against SETUP_USER=root and rewrite README for fresh VM setup Script now exits early with a clear error if run as root directly instead of via sudo, preventing silent misconfiguration where everything ends up under /root. README updated: added VM specs (2vCPU/4GB/20GB), clarified sudo vs su- requirement, replaced stale crontab references with systemd commands, corrected log path (/tmp/hamcp.log -> /var/log/hamcp-demo.log), and added full reset procedure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(internal): extend HA wait to 15 min and make timeout a warning not an error On first install the HA image (~600MB) must be pulled before the container starts, which takes ~10 minutes. The previous 3-minute wait caused a false failure exit. Now waits up to 15 minutes (180 × 5s) and on timeout emits a warning instead of exiting 1, since the service continues running and HA will come up on its own. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(internal): address Gemini review comments on update service and logging - Run update service as root with sudo -u for git pull (cleaner than sudoers rule; ExecStartPost can then call systemctl restart directly without privilege escalation) - Add network-online.target dependency to update service (git pull needs network) - Remove sudoers rule (no longer needed now update service runs as root) - Drop file logging from both services (systemd journal handles rotation automatically; unbounded /var/log/hamcp-demo.log was a disk exhaustion risk) - Add pkill on migration to clean up any leaked hamcp-test-env processes from old setup - Update README and success output to remove stale log file references Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(internal): address second round of Gemini review comments - Remove Requires= from timer unit (caused service to trigger on timer start, defeating the OnCalendar schedule; timer auto-triggers matching service by name) - Use sudo -i -u instead of sudo -u for git pull (ensures $HOME is set correctly) - Add docker image prune -af to update service (old cron pruned images; without this unused images accumulate and fill disk over time) - Remove stale sudoers rule step from README and renumber (rule was dropped in previous commit when update service moved to running as root) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(internal): use image column to find container name in success banner The ancestor filter returns empty immediately after systemctl restart because Docker hasn't finished registering image metadata for the new container. Grep the image column instead, which is always populated. * fix(internal): send SIGINT to hamcp-test-env on service stop hamcp-test-env only handles KeyboardInterrupt (SIGINT) for cleanup. systemd's default stop signal is SIGTERM, which Python does not convert to KeyboardInterrupt, so the HA container was left running after restart. KillSignal=SIGINT ensures the cleanup path runs on every stop/restart. * fix(internal): avoid creating empty crontab on fresh hosts during migration On a host with no existing crontab, crontab -l exits 1 with empty stdout, grep -v outputs nothing, and crontab - installs a blank crontab as a side effect. Only rewrite the crontab when the old hamcp-test-env entries are actually present. * fix(internal): add Wants=network-online.target and stop before cleanup Two fixes from Patch76 follow-up: - Add Wants=network-online.target to hamcp-demo.service so the unit pulls the target on cold boot (After= alone only orders, doesn't pull) - systemctl stop hamcp-demo before pkill on re-runs, preventing systemd from auto-restarting the unit while cleanup is in progress * fix(internal): use sudo -i -u for uv install to ensure correct \$HOME sudo -u without -i doesn't reset \$HOME when always_set_home is off, causing uv to install to /root/.local/bin instead of the target user's home. The check for an existing install then always fails on those hosts. * fix(internal): handle pipefail on crontab migration and fix README - Wrap grep -v in { ... || true; } so pipefail doesn't abort when the legacy crontab contains only hamcp-test-env lines (grep exits 1 with no output, which is exactly the migration target case) - Clarify README timing: 3-10 min on re-runs, up to 15 min on first install when the ~600MB HA image must be pulled - Fix full-reset snippet: filter by ancestor image instead of stopping all containers, matching what step 9 of the script already does * fix(internal): run demo update timer daily instead of weekly Daily git pull ensures the demo server picks up new releases within 24h rather than waiting up to 7 days. * docs(internal): update README timer frequency to daily --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent d4d9d0a commit 2fbd6a8

2 files changed

Lines changed: 152 additions & 76 deletions

File tree

tests/lab-setup/README.md

Lines changed: 64 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -2,95 +2,109 @@
22

33
This directory contains scripts for setting up a persistent Home Assistant test environment, useful for demos, development, and integration testing.
44

5+
## Requirements
6+
7+
- Ubuntu/Debian Linux server (tested on Debian 12)
8+
- Minimum specs: **2 vCPU, 4GB RAM, 20GB disk** (Home Assistant uses ~1.5GB RAM; 6GB swap is created automatically for headroom)
9+
- A non-root user with `sudo` access
10+
- Domain name pointing to the server's IP (for HTTPS; optional for local-only)
11+
- Ports **80** and **443** open inbound (for Caddy + Let's Encrypt)
12+
513
## Quick Start
614

715
```bash
8-
# Download and run on a fresh Ubuntu/Debian server
16+
# 1. SSH in as a non-root user (e.g. the default GCP/AWS/Azure user)
17+
# 2. Download and run:
918
curl -fsSL https://raw.githubusercontent.com/homeassistant-ai/ha-mcp/master/tests/lab-setup/setup-ha-mcp.sh -o setup-ha-mcp.sh
1019
chmod +x setup-ha-mcp.sh
1120
sudo ./setup-ha-mcp.sh your-domain.example.com
1221
```
1322

23+
> **Important:** Use `sudo ./setup-ha-mcp.sh`, not `sudo su -` followed by `./setup-ha-mcp.sh`.
24+
> The script uses `$SUDO_USER` to identify which user to configure. Running as root directly loses that information and the script will exit with an error.
25+
>
26+
> If calling from a **root cron job**, set it explicitly:
27+
> `SUDO_USER=youruser ./setup-ha-mcp.sh your-domain.example.com`
28+
29+
Setup takes about 3–10 minutes on subsequent runs. Allow up to 15 minutes on first install when the ~600 MB HA image must be pulled. Home Assistant starts automatically.
30+
1431
## What It Does
1532

1633
The setup script is **idempotent** (safe to re-run) and performs:
1734

18-
1. **Swap Configuration** - Creates 6GB swap for small VMs
19-
2. **Package Installation** - curl, git, ca-certificates, gnupg
20-
3. **Docker Installation** - Via official get.docker.com script
21-
4. **uv Installation** - Python package manager for running ha-mcp
22-
5. **ha-mcp Clone** - Clones the repository to `~/ha-mcp`
23-
6. **Crontab Setup** - Auto-starts on reboot + weekly reset (Mondays 3am)
24-
7. **Caddy Reverse Proxy** - HTTPS with automatic Let's Encrypt certificates
25-
8. **Unattended Upgrades** - Auto-updates system packages with auto-reboot at 4am
26-
9. **Container Cleanup** - Removes old HA containers
27-
10. **Start Test Environment** - Launches hamcp-test-env in background
28-
29-
## Weekly Reset (Monday 3am)
30-
31-
The lab environment automatically resets every Monday at 3am:
32-
- Pulls latest ha-mcp changes from git
33-
- Stops and removes Home Assistant containers
34-
- Prunes unused Docker images (prevents disk fill)
35-
- Restarts hamcp-test-env with fresh container
35+
1. **Swap** — Creates 6GB swap for small VMs
36+
2. **Packages** — Installs curl, git, ca-certificates, gnupg
37+
3. **Docker** — Via official get.docker.com script
38+
4. **uv** — Python package manager for running ha-mcp
39+
5. **ha-mcp repo** — Clones to `~/ha-mcp` (or pulls if it already exists)
40+
6. **Systemd service** — Creates `hamcp-demo.service` (starts on boot, restarts on failure) and `hamcp-demo-update.timer` (daily at 3am: git pull, docker image prune, service restart)
41+
7. **Caddy** — Reverse proxy with automatic Let's Encrypt TLS for your domain
42+
8. **Unattended upgrades** — Auto-updates OS packages, reboots at 4am if needed
43+
9. **Container cleanup** — Removes stale HA containers and any leaked processes
44+
10. **Start** — Launches `hamcp-demo` via systemd and waits for Home Assistant to become ready
3645

37-
## Requirements
46+
## Access
3847

39-
- Ubuntu/Debian Linux server
40-
- Root access (sudo)
41-
- Domain pointing to server (for HTTPS)
42-
- Ports 80/443 open (for Caddy/HTTPS)
43-
- Port 8123 (internal, for Home Assistant)
48+
After setup:
4449

45-
## Usage
50+
| | URL |
51+
|---|---|
52+
| Local | http://localhost:8123 |
53+
| External | https://your-domain.example.com |
54+
| Credentials | `dev` / `dev` |
4655

47-
```bash
48-
# With custom domain (HTTPS enabled)
49-
sudo ./setup-ha-mcp.sh my-ha-lab.example.com
56+
## Managing the Service
5057

51-
# Without domain (local access only)
52-
sudo ./setup-ha-mcp.sh ""
53-
```
58+
```bash
59+
# Status
60+
sudo systemctl status hamcp-demo
5461

55-
## Access
62+
# Restart (e.g. after manual code changes)
63+
sudo systemctl restart hamcp-demo
5664

57-
After setup:
65+
# Live logs
66+
sudo journalctl -u hamcp-demo -f
5867

59-
| Access | URL |
60-
|--------|-----|
61-
| Local | http://localhost:8123 |
62-
| External | https://your-domain.example.com |
63-
| Credentials | dev / dev |
68+
# Weekly update timer — next run and last result
69+
sudo systemctl list-timers hamcp-demo-update.timer
70+
sudo journalctl -u hamcp-demo-update --no-pager -n 20
71+
```
6472

6573
## Logs
6674

6775
```bash
76+
# Service log (startup, HA output, errors)
77+
sudo journalctl -u hamcp-demo -f
78+
6879
# Home Assistant container logs
6980
docker logs -f $(docker ps --filter "ancestor=ghcr.io/home-assistant/home-assistant" -q)
70-
71-
# Startup script log
72-
tail -f /tmp/hamcp.log
7381
```
7482

7583
## Troubleshooting
7684

77-
### Container not starting
85+
### Home Assistant not starting
7886
```bash
79-
cat /tmp/hamcp.log
87+
sudo journalctl -u hamcp-demo --no-pager -n 50
8088
docker ps -a
8189
```
8290

8391
### Caddy certificate issues
8492
```bash
85-
sudo journalctl -u caddy -f
86-
sudo caddy validate --config /etc/caddy/Caddyfile
93+
sudo journalctl -u caddy --no-pager -n 30
94+
# Force renewal by restarting Caddy
95+
sudo systemctl restart caddy
8796
```
8897

8998
### Restart the environment
9099
```bash
91-
# Stop
92-
docker stop $(docker ps --filter "ancestor=ghcr.io/home-assistant/home-assistant" -q)
100+
sudo systemctl restart hamcp-demo
101+
```
93102

94-
# Start
95-
cd ~/ha-mcp && HA_TEST_PORT=8123 ~/.local/bin/uv run hamcp-test-env --no-interactive &
103+
### Full reset (re-clone and restart from scratch)
104+
```bash
105+
sudo systemctl stop hamcp-demo
106+
docker ps -aq --filter "ancestor=ghcr.io/home-assistant/home-assistant" | xargs -r docker rm -f 2>/dev/null || true
107+
docker ps -aq --filter "ancestor=testcontainers/ryuk" | xargs -r docker rm -f 2>/dev/null || true
108+
rm -rf ~/ha-mcp
109+
sudo ./setup-ha-mcp.sh your-domain.example.com
96110
```

tests/lab-setup/setup-ha-mcp.sh

Lines changed: 88 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ error() { echo -e "${RED}[ERROR]${NC} $1"; exit 1; }
2222

2323
#=============================================================================
2424
[[ $EUID -ne 0 ]] && error "Run as root: sudo $0 [domain]"
25+
[[ "$SETUP_USER" == "root" ]] && error "Do not run as root directly. Use: sudo $0 [domain]\nIf calling from a root cron job, set: SUDO_USER=youruser $0 [domain]"
2526
info "Setting up ha-mcp test env for user: $SETUP_USER"
2627
info "Domain: $DOMAIN"
2728

@@ -66,7 +67,7 @@ fi
6667
# 4. UV
6768
if [[ ! -f "$UV_PATH" ]]; then
6869
info "Installing uv..."
69-
sudo -u "$SETUP_USER" bash -c 'curl -LsSf https://astral.sh/uv/install.sh | sh'
70+
sudo -i -u "$SETUP_USER" bash -c 'curl -LsSf https://astral.sh/uv/install.sh | sh'
7071
else
7172
info "uv already installed"
7273
fi
@@ -82,15 +83,70 @@ else
8283
fi
8384

8485
#=============================================================================
85-
# 6. CRONTAB (startup + weekly reset)
86-
info "Setting up crontab..."
87-
CRON_REBOOT="@reboot sleep 10 && cd $SETUP_HOME/ha-mcp && HA_TEST_PORT=$HA_PORT $UV_PATH run hamcp-test-env --no-interactive >> /tmp/hamcp.log 2>&1"
88-
CRON_WEEKLY="0 3 * * 1 cd $SETUP_HOME/ha-mcp && git pull --ff-only && docker stop \$(docker ps -q --filter ancestor=ghcr.io/home-assistant/home-assistant) 2>/dev/null; docker rm \$(docker ps -aq --filter ancestor=ghcr.io/home-assistant/home-assistant) 2>/dev/null; docker image prune -af 2>/dev/null; HA_TEST_PORT=$HA_PORT $UV_PATH run hamcp-test-env --no-interactive >> /tmp/hamcp.log 2>&1"
89-
(
90-
sudo -u "$SETUP_USER" crontab -l 2>/dev/null | grep -v "hamcp-test-env" || true
91-
echo "$CRON_REBOOT"
92-
echo "$CRON_WEEKLY"
93-
) | sudo -u "$SETUP_USER" crontab -
86+
# 6. SYSTEMD SERVICE (replaces cron - ensures only one instance runs, auto-restarts)
87+
info "Setting up systemd service..."
88+
89+
# Remove old cron entries if they exist (migration from cron-based setup)
90+
if sudo -u "$SETUP_USER" crontab -l 2>/dev/null | grep -q "hamcp-test-env"; then
91+
sudo -u "$SETUP_USER" crontab -l 2>/dev/null | { grep -v "hamcp-test-env" || true; } | sudo -u "$SETUP_USER" crontab -
92+
fi
93+
94+
cat > /etc/systemd/system/hamcp-demo.service << SVCEOF
95+
[Unit]
96+
Description=HA-MCP Demo Test Environment
97+
After=docker.service network-online.target
98+
Wants=network-online.target
99+
Requires=docker.service
100+
101+
[Service]
102+
Type=simple
103+
User=${SETUP_USER}
104+
Group=${SETUP_USER}
105+
WorkingDirectory=${SETUP_HOME}/ha-mcp
106+
Environment=HA_TEST_PORT=${HA_PORT}
107+
ExecStart=${UV_PATH} run hamcp-test-env --no-interactive
108+
Restart=on-failure
109+
RestartSec=30s
110+
KillSignal=SIGINT
111+
TimeoutStopSec=60
112+
113+
[Install]
114+
WantedBy=multi-user.target
115+
SVCEOF
116+
117+
cat > /etc/systemd/system/hamcp-demo-update.service << SVCEOF
118+
[Unit]
119+
Description=HA-MCP Demo Weekly Update
120+
After=network-online.target docker.service
121+
Wants=network-online.target
122+
123+
[Service]
124+
Type=oneshot
125+
ExecStart=/usr/bin/sudo -i -u ${SETUP_USER} /usr/bin/git -C ${SETUP_HOME}/ha-mcp pull --ff-only
126+
ExecStart=/usr/bin/docker image prune -af
127+
ExecStartPost=/usr/bin/systemctl restart hamcp-demo
128+
SVCEOF
129+
130+
cat > /etc/systemd/system/hamcp-demo-update.timer << SVCEOF
131+
[Unit]
132+
Description=HA-MCP Demo Weekly Update Timer
133+
134+
[Timer]
135+
OnCalendar=*-*-* 03:00:00
136+
AccuracySec=1h
137+
Persistent=true
138+
139+
[Install]
140+
WantedBy=timers.target
141+
SVCEOF
142+
143+
# Remove sudoers rule if it exists from a previous install (no longer needed)
144+
rm -f /etc/sudoers.d/hamcp-demo
145+
146+
systemctl daemon-reload
147+
systemctl enable hamcp-demo.service
148+
systemctl enable hamcp-demo-update.timer
149+
systemctl start hamcp-demo-update.timer
94150

95151
#=============================================================================
96152
# 7. CADDY
@@ -146,32 +202,38 @@ APT::Periodic::AutocleanInterval "7";
146202
AUTOEOF
147203

148204
#=============================================================================
149-
# 9. STOP OLD CONTAINERS
150-
info "Cleaning up old containers..."
205+
# 9. STOP OLD CONTAINERS + PROCESSES
206+
info "Cleaning up old containers and processes..."
207+
systemctl stop hamcp-demo 2>/dev/null || true
151208
docker ps -aq --filter "ancestor=ghcr.io/home-assistant/home-assistant" | xargs -r docker rm -f 2>/dev/null || true
209+
docker ps -aq --filter "ancestor=testcontainers/ryuk" | xargs -r docker rm -f 2>/dev/null || true
210+
pkill -u "$SETUP_USER" -f "hamcp-test-env" 2>/dev/null || true
152211

153212
#=============================================================================
154-
# 10. START HA-MCP
155-
info "Starting hamcp-test-env..."
156-
sudo -u "$SETUP_USER" sg docker -c "cd $SETUP_HOME/ha-mcp && HA_TEST_PORT=$HA_PORT $UV_PATH run hamcp-test-env --no-interactive > /tmp/hamcp.log 2>&1 &"
213+
# 10. START HA-MCP VIA SYSTEMD
214+
info "Starting hamcp-demo service..."
215+
systemctl restart hamcp-demo.service
157216

158217
#=============================================================================
159218
# 11. WAIT FOR HA
160-
info "Waiting for Home Assistant to start..."
161-
for i in {1..60}; do
219+
# On first install, the HA image (~600MB) must be pulled before the container
220+
# starts. Allow up to 15 minutes to cover both pull and boot time.
221+
info "Waiting for Home Assistant to start (up to 15 minutes on first install)..."
222+
HA_READY=0
223+
for i in {1..180}; do
162224
if curl -s -o /dev/null -w "%{http_code}" "http://localhost:$HA_PORT" 2>/dev/null | grep -qE "200|401"; then
225+
HA_READY=1
163226
break
164227
fi
165228
echo -n "."
166-
sleep 2
229+
sleep 5
167230
done
168231
echo ""
169232

170233
#=============================================================================
171-
# 11. VERIFY
172-
sleep 3
173-
if docker ps | grep -q "home-assistant"; then
174-
CONTAINER=$(docker ps --filter "ancestor=ghcr.io/home-assistant/home-assistant" --format "{{.Names}}" | head -1)
234+
# 12. VERIFY
235+
if [[ $HA_READY -eq 1 ]]; then
236+
CONTAINER=$(docker ps --format "{{.Image}}\t{{.Names}}" | awk -F'\t' '/home-assistant/{print $2}' | head -1)
175237
echo ""
176238
echo "=============================================="
177239
echo -e "${GREEN}Setup Complete!${NC}"
@@ -183,11 +245,11 @@ if docker ps | grep -q "home-assistant"; then
183245
echo "Credentials: dev / dev"
184246
echo ""
185247
echo "Logs: docker logs -f $CONTAINER"
186-
echo "Startup log: tail -f /tmp/hamcp.log"
248+
echo "Service log: journalctl -u hamcp-demo -f"
187249
echo "=============================================="
188250
else
189251
echo ""
190-
echo -e "${RED}Container not running!${NC}"
191-
echo "Check logs: cat /tmp/hamcp.log"
192-
exit 1
252+
warn "Home Assistant did not respond within 15 minutes."
253+
warn "The service is running — HA may still be starting (check: journalctl -u hamcp-demo -f)"
254+
warn "If it stays down: journalctl -u hamcp-demo --no-pager -n 50"
193255
fi

0 commit comments

Comments
 (0)