Skip to content

Commit fc8bae7

Browse files
vjgit96erichare
vjgit96
authored and
erichare
committed
feat: add DB migration validation workflow (LE-1259) [backport 1.9.4] (#13348)
* feat: Add DB migration validation workflow for nightly builds (LE-1259) - Implements automated DB migration testing for nightly builds - Tests two scenarios: pip/venv and Docker Compose migrations - Validates migration from stable to nightly versions - Verifies data persistence (witness flows) across migrations - Integrated into nightly_build.yml workflow - Includes Slack notifications for migration test results This addresses the critical blocker identified by QA team for ensuring safe database migrations in production deployments. * docs: Add DB migration validation documentation - Comprehensive guide for LE-1259 implementation - Detailed test scenarios and execution details - Environment configuration and success criteria - Monitoring and troubleshooting guidelines - Placed in docs/docs/Deployment/ for easy access * fix: address PR review comments for DB migration validation - Use actual nightly tag from create-nightly-tag output instead of hardcoded :latest - Wire POSTGRES_VERSION env var into postgres service image tag - Remove unnecessary checkout steps from both migration jobs - Update step name from 'Create witness flow and credentials' to 'Create witness flow' - Add -f flag to curl commands for fail-fast behavior - Add flow creation verification with error handling - Fix version extraction logic to properly test nightly build instead of PyPI latest - Remove deprecated docker-compose version field - Remove duplicate Slack notification job (consolidated in nightly_build.yml) Addresses all 9 issues identified by @ogabrielluiz in PR review * docs: remove implementation summary from user-facing docs Per @ogabrielluiz review feedback, this file reads as an implementation summary (Jira ticket, branch name, 'Next Steps', 'Files Changed: 2') rather than user-facing documentation. The Deployment section is for end-user docs, and this content is better suited for the PR description. Also not added to sidebars.js, so would be an orphan page. * fix(workflows): address 6 issues from Gabriel's second review of DB migration validation Fixes all remaining issues identified in PR #13249 review: 1. Remove schedule trigger - only works on default branch, would cause duplicate runs 2. Fix postgres service image - hardcode to postgres:16 (env context not available in services) 3. Add curl fail-fast flags - use -fsSL for immediate failure on errors 4. Add flow ID verification - check witness data creation succeeded before proceeding 5. Remove orphaned Slack JSON - cleanup leftover from removed notify-results job 6. Fix version extraction - strip 'v' prefix for pip install (${VERSION#v}) 7. Fix Docker image reference in nightly_build.yml - pass full image path with tag 8. Fix success notification - check migration validation didn't fail All changes validated locally: - YAML syntax validation passed - Docker Compose config validated - Tag manipulation logic tested (v prefix handling) - Curl command structure verified - Test credentials marked with pragma allowlist secret comments Related: LE-1259, PR #13249 Depends on: PR #13212 (Docker volume permissions fix)
1 parent ca6cffc commit fc8bae7

2 files changed

Lines changed: 415 additions & 4 deletions

File tree

‎.github/workflows/db-migration-validation.yml‎

Lines changed: 397 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,397 @@
1+
name: DB Migration Validation
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
nightly_tag:
7+
description: "Nightly tag to test migration to"
8+
required: true
9+
type: string
10+
workflow_dispatch:
11+
inputs:
12+
nightly_tag:
13+
description: "Nightly tag to test migration to (e.g., langflowai/langflow-nightly:latest)"
14+
required: false
15+
type: string
16+
default: "langflowai/langflow-nightly:latest"
17+
# Note: This workflow is called by nightly_build.yml after Docker images are built
18+
19+
env:
20+
PYTHON_VERSION: "3.13"
21+
POSTGRES_DB: langflow_test
22+
POSTGRES_USER: langflow
23+
POSTGRES_PASSWORD: langflow_test_pass # pragma: allowlist secret
24+
25+
jobs:
26+
migration-pip-venv:
27+
name: "Migration Test: pip/venv (stable → nightly)"
28+
runs-on: ubuntu-latest
29+
timeout-minutes: 30
30+
31+
services:
32+
postgres:
33+
image: postgres:16
34+
env:
35+
POSTGRES_DB: ${{ env.POSTGRES_DB }}
36+
POSTGRES_USER: ${{ env.POSTGRES_USER }}
37+
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
38+
options: >-
39+
--health-cmd pg_isready
40+
--health-interval 10s
41+
--health-timeout 5s
42+
--health-retries 5
43+
ports:
44+
- 5432:5432
45+
46+
steps:
47+
- name: Setup Python
48+
uses: actions/setup-python@v6
49+
with:
50+
python-version: ${{ env.PYTHON_VERSION }}
51+
52+
- name: Install uv
53+
uses: astral-sh/setup-uv@v6
54+
with:
55+
enable-cache: false
56+
57+
- name: Create test directory
58+
run: |
59+
mkdir -p migration-test
60+
cd migration-test
61+
62+
- name: Install latest stable Langflow with PostgreSQL
63+
working-directory: migration-test
64+
run: |
65+
echo "Installing latest stable langflow[postgresql]..."
66+
uv venv
67+
source .venv/bin/activate
68+
uv pip install "langflow[postgresql]"
69+
70+
# Verify installation
71+
python -c "import langflow; print(f'Installed Langflow version: {langflow.__version__}')"
72+
73+
- name: Initialize database with stable version
74+
working-directory: migration-test
75+
env:
76+
LANGFLOW_DATABASE_URL: postgresql://${{ env.POSTGRES_USER }}:${{ env.POSTGRES_PASSWORD }}@localhost:5432/${{ env.POSTGRES_DB }} # pragma: allowlist secret
77+
LANGFLOW_SUPERUSER: admin
78+
LANGFLOW_SUPERUSER_PASSWORD: admin123 # pragma: allowlist secret
79+
run: |
80+
source .venv/bin/activate
81+
82+
echo "Starting Langflow to initialize database..."
83+
timeout 120 bash -c '
84+
python -m langflow run --host 127.0.0.1 --port 7860 --backend-only &
85+
LANGFLOW_PID=$!
86+
until curl -f http://127.0.0.1:7860/health_check 2>/dev/null; do
87+
sleep 2
88+
done
89+
kill $LANGFLOW_PID
90+
wait $LANGFLOW_PID 2>/dev/null || true
91+
' || {
92+
echo "Failed to start Langflow stable version"
93+
exit 1
94+
}
95+
96+
echo "Database initialized successfully with stable version"
97+
98+
- name: Create witness flow
99+
working-directory: migration-test
100+
env:
101+
LANGFLOW_DATABASE_URL: postgresql://${{ env.POSTGRES_USER }}:${{ env.POSTGRES_PASSWORD }}@localhost:5432/${{ env.POSTGRES_DB }}
102+
run: |
103+
source .venv/bin/activate
104+
105+
# Start Langflow briefly to create test data
106+
python -m langflow run --host 127.0.0.1 --port 7860 --backend-only &
107+
LANGFLOW_PID=$!
108+
109+
# Wait for startup
110+
timeout 60 bash -c 'until curl -f http://127.0.0.1:7860/health_check 2>/dev/null; do sleep 2; done'
111+
112+
# Create a simple flow via API
113+
curl -fsSL -X POST http://127.0.0.1:7860/api/v1/flows/ \
114+
-H "Content-Type: application/json" \
115+
-d '{
116+
"name": "Migration Witness Flow",
117+
"description": "Test flow to verify data persistence across migration",
118+
"data": {"nodes": [], "edges": []}
119+
}' > flow_response.json
120+
121+
# Verify flow was created
122+
if ! grep -q '"id"' flow_response.json; then
123+
echo "❌ Flow creation failed - no id in response"
124+
cat flow_response.json
125+
exit 1
126+
fi
127+
echo "✅ Witness flow created successfully"
128+
129+
# Stop Langflow
130+
kill $LANGFLOW_PID
131+
wait $LANGFLOW_PID 2>/dev/null || true
132+
133+
echo "Witness data created"
134+
135+
- name: Upgrade to nightly version
136+
working-directory: migration-test
137+
run: |
138+
source .venv/bin/activate
139+
140+
NIGHTLY_TAG="${{ inputs.nightly_tag || 'langflowai/langflow-nightly:latest' }}"
141+
echo "Upgrading to nightly version: $NIGHTLY_TAG"
142+
143+
# Extract version from Docker tag (format: langflowai/langflow-nightly:v1.10.0.dev20260522)
144+
if [[ "$NIGHTLY_TAG" == *":"* ]]; then
145+
VERSION="${NIGHTLY_TAG##*:}"
146+
echo "Extracted version from tag: $VERSION"
147+
148+
# Strip 'v' prefix if present (PyPI doesn't use 'v' prefix)
149+
VERSION="${VERSION#v}"
150+
echo "Version for PyPI: $VERSION"
151+
152+
if [[ "$VERSION" == "latest" ]]; then
153+
# Install latest nightly from PyPI
154+
uv pip install --upgrade langflow-nightly[postgresql]
155+
else
156+
# Install specific version
157+
uv pip install --upgrade "langflow-nightly[postgresql]==$VERSION"
158+
fi
159+
else
160+
# Direct version string (strip 'v' prefix if present)
161+
VERSION="${NIGHTLY_TAG#v}"
162+
uv pip install --upgrade "langflow-nightly[postgresql]==$VERSION"
163+
fi
164+
165+
# Verify upgrade
166+
python -c "import langflow; print(f'Upgraded to Langflow version: {langflow.__version__}')"
167+
168+
- name: Run migration and verify startup
169+
working-directory: migration-test
170+
env:
171+
LANGFLOW_DATABASE_URL: postgresql://${{ env.POSTGRES_USER }}:${{ env.POSTGRES_PASSWORD }}@localhost:5432/${{ env.POSTGRES_DB }} # pragma: allowlist secret
172+
LANGFLOW_SUPERUSER: admin
173+
LANGFLOW_SUPERUSER_PASSWORD: admin123 # pragma: allowlist secret
174+
run: |
175+
source .venv/bin/activate
176+
177+
echo "Starting Langflow nightly to run migrations..."
178+
timeout 180 bash -c '
179+
python -m langflow run --host 127.0.0.1 --port 7860 --backend-only > langflow_nightly.log 2>&1 &
180+
LANGFLOW_PID=$!
181+
182+
until curl -f http://127.0.0.1:7860/health_check 2>/dev/null; do
183+
if ! kill -0 $LANGFLOW_PID 2>/dev/null; then
184+
echo "Langflow process died during startup"
185+
cat langflow_nightly.log
186+
exit 1
187+
fi
188+
sleep 2
189+
done
190+
191+
echo "Langflow nightly started successfully"
192+
kill $LANGFLOW_PID
193+
wait $LANGFLOW_PID 2>/dev/null || true
194+
' || {
195+
echo "Failed to start Langflow nightly version"
196+
cat langflow_nightly.log || true
197+
exit 1
198+
}
199+
200+
- name: Verify witness data persisted
201+
working-directory: migration-test
202+
env:
203+
LANGFLOW_DATABASE_URL: postgresql://${{ env.POSTGRES_USER }}:${{ env.POSTGRES_PASSWORD }}@localhost:5432/${{ env.POSTGRES_DB }}
204+
run: |
205+
source .venv/bin/activate
206+
207+
# Start Langflow to query data
208+
python -m langflow run --host 127.0.0.1 --port 7860 --backend-only &
209+
LANGFLOW_PID=$!
210+
211+
timeout 60 bash -c 'until curl -f http://127.0.0.1:7860/health_check 2>/dev/null; do sleep 2; done'
212+
213+
# Verify witness flow exists
214+
curl -f http://127.0.0.1:7860/api/v1/flows/ > flows_after_migration.json
215+
216+
if grep -q "Migration Witness Flow" flows_after_migration.json; then
217+
echo "✅ Witness flow found after migration"
218+
else
219+
echo "❌ Witness flow NOT found after migration"
220+
cat flows_after_migration.json
221+
kill $LANGFLOW_PID
222+
exit 1
223+
fi
224+
225+
kill $LANGFLOW_PID
226+
wait $LANGFLOW_PID 2>/dev/null || true
227+
228+
- name: Upload logs on failure
229+
if: failure()
230+
uses: actions/upload-artifact@v6
231+
with:
232+
name: migration-pip-venv-logs
233+
path: |
234+
migration-test/*.log
235+
migration-test/*.json
236+
retention-days: 7
237+
238+
migration-docker-compose:
239+
name: "Migration Test: Docker Compose (stable → nightly)"
240+
runs-on: ubuntu-latest
241+
timeout-minutes: 30
242+
243+
steps:
244+
- name: Checkout code
245+
uses: actions/checkout@v6
246+
247+
- name: Create Docker Compose test directory
248+
run: |
249+
mkdir -p docker-migration-test
250+
cd docker-migration-test
251+
252+
- name: Create Docker Compose file for stable
253+
working-directory: docker-migration-test
254+
run: |
255+
cat > docker-compose.yml <<'EOF'
256+
version: "3.8"
257+
258+
services:
259+
langflow:
260+
image: langflowai/langflow:latest
261+
ports:
262+
- "7860:7860"
263+
environment: # pragma: allowlist secret
264+
- LANGFLOW_DATABASE_URL=postgresql://langflow:langflow@postgres:5432/langflow # pragma: allowlist secret
265+
- LANGFLOW_SUPERUSER=admin
266+
- LANGFLOW_SUPERUSER_PASSWORD=admin123 # pragma: allowlist secret
267+
depends_on:
268+
postgres:
269+
condition: service_healthy
270+
healthcheck:
271+
test: ["CMD", "curl", "-f", "http://localhost:7860/health_check"]
272+
interval: 10s
273+
timeout: 5s
274+
retries: 10
275+
276+
postgres:
277+
image: postgres:16
278+
environment: # pragma: allowlist secret
279+
- POSTGRES_USER=langflow
280+
- POSTGRES_PASSWORD=langflow # pragma: allowlist secret
281+
- POSTGRES_DB=langflow
282+
volumes:
283+
- langflow_postgres_data:/var/lib/postgresql/data
284+
healthcheck:
285+
test: ["CMD-SHELL", "pg_isready -U langflow"]
286+
interval: 10s
287+
timeout: 5s
288+
retries: 5
289+
290+
volumes:
291+
langflow_postgres_data:
292+
EOF
293+
294+
- name: Start stable Langflow with Docker Compose
295+
working-directory: docker-migration-test
296+
run: |
297+
echo "Starting Langflow stable version..."
298+
docker compose up -d
299+
300+
echo "Waiting for Langflow to be healthy..."
301+
timeout 180 bash -c 'until docker compose exec -T langflow curl -f http://localhost:7860/health_check 2>/dev/null; do sleep 5; done' || {
302+
echo "Langflow stable failed to start"
303+
docker compose logs
304+
exit 1
305+
}
306+
307+
echo "Langflow stable is running"
308+
309+
- name: Create witness flow via API
310+
working-directory: docker-migration-test
311+
run: |
312+
echo "Creating witness flow..."
313+
curl -fsSL -X POST http://localhost:7860/api/v1/flows/ \
314+
-H "Content-Type: application/json" \
315+
-d '{
316+
"name": "Docker Migration Witness Flow",
317+
"description": "Test flow for Docker Compose migration",
318+
"data": {"nodes": [], "edges": []}
319+
}' > flow_response.json
320+
321+
# Verify flow was created
322+
if ! grep -q '"id"' flow_response.json; then
323+
echo "❌ Flow creation failed - no id in response"
324+
cat flow_response.json
325+
exit 1
326+
fi
327+
echo "✅ Witness flow created successfully"
328+
329+
- name: Stop stable Langflow (keep PostgreSQL volume)
330+
working-directory: docker-migration-test
331+
run: |
332+
echo "Stopping Langflow stable..."
333+
docker compose stop langflow
334+
docker compose rm -f langflow
335+
336+
- name: Update to nightly image
337+
working-directory: docker-migration-test
338+
run: |
339+
NIGHTLY_TAG="${{ inputs.nightly_tag || 'langflowai/langflow-nightly:latest' }}"
340+
echo "Updating to nightly: $NIGHTLY_TAG"
341+
342+
# Update docker-compose.yml to use nightly image
343+
sed -i "s|image: langflowai/langflow:latest|image: $NIGHTLY_TAG|" docker-compose.yml
344+
345+
cat docker-compose.yml
346+
347+
- name: Start nightly Langflow with same PostgreSQL volume
348+
working-directory: docker-migration-test
349+
run: |
350+
echo "Starting Langflow nightly version..."
351+
docker compose up -d langflow
352+
353+
echo "Waiting for Langflow nightly to be healthy..."
354+
timeout 180 bash -c 'until docker compose exec -T langflow curl -f http://localhost:7860/health_check 2>/dev/null; do sleep 5; done' || {
355+
echo "Langflow nightly failed to start"
356+
docker compose logs langflow
357+
exit 1
358+
}
359+
360+
echo "Langflow nightly started successfully"
361+
362+
- name: Verify witness data persisted
363+
working-directory: docker-migration-test
364+
run: |
365+
echo "Verifying witness flow..."
366+
curl -f http://localhost:7860/api/v1/flows/ > flows_after_migration.json
367+
368+
if grep -q "Docker Migration Witness Flow" flows_after_migration.json; then
369+
echo "✅ Witness flow found after Docker migration"
370+
else
371+
echo "❌ Witness flow NOT found after Docker migration"
372+
cat flows_after_migration.json
373+
exit 1
374+
fi
375+
376+
- name: Collect logs on failure
377+
if: failure()
378+
working-directory: docker-migration-test
379+
run: |
380+
docker compose logs > docker-compose-logs.txt
381+
382+
- name: Upload logs on failure
383+
if: failure()
384+
uses: actions/upload-artifact@v6
385+
with:
386+
name: migration-docker-compose-logs
387+
path: |
388+
docker-migration-test/*.log
389+
docker-migration-test/*.json
390+
docker-migration-test/*.txt
391+
retention-days: 7
392+
393+
- name: Cleanup
394+
if: always()
395+
working-directory: docker-migration-test
396+
run: |
397+
docker compose down -v

0 commit comments

Comments
 (0)