feat(ci): add manual migration workflow with multi-user stress - #242
Closed
lice-reis wants to merge 7 commits into
Closed
feat(ci): add manual migration workflow with multi-user stress#242lice-reis wants to merge 7 commits into
lice-reis wants to merge 7 commits into
Conversation
New on-demand workflow `migration-manual.yml`. Designed for release-grade migration validation between arbitrary source/target images. Inputs: - source_image / target_image (full image path; supports both langflowai/langflow and langflowai/langflow-nightly, including mixed repo migrations like stable → nightly) - database: postgresql | sqlite | both - workers: LANGFLOW_WORKERS for PostgreSQL (SQLite forces 1) - concurrent_users: distinct user accounts created and stressed in parallel - stress_duration_seconds: post-migration stress duration Per matrix entry: 1. Source image boots clean, creates N test users + 1 witness flow 2. Target image boots on the same DB (alembic migration runs) 3. Verify witness flow + all N users preserved 4. N users log in concurrently and hit auth + flow-list endpoints for the configured duration (designed to reproduce the runtime conditions of langflow-ai/langflow#13157) 5. Scan target log for the five DB/session error patterns reported in that issue; fail the job if any match SQLite is intentionally pinned to a single worker because SQLite's own documentation (sqlite.org/howtocorrupt.html §2.7, §8.1) describes multi-worker on SQLite as a known corruption vector independent of the application — running it would produce noise rather than meaningful regression signal. Trigger: workflow_dispatch only (no schedule). Intended for release cycles, not daily CI. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…rage Extends migration-manual.yml to verify what migration-test.yml already exercises (real flow + encrypted credential) plus the paced flow runs during stress. What the test now covers, per matrix entry: - Source seeds the DB with: N users, the Simple Agent starter project instantiated as a flow, and an OPENAI_API_KEY Variable (Fernet-encrypted using LANGFLOW_SECRET_KEY). - Target boots on the same DB. Verifies the flow, all N users, AND the variable row all survived migration. - Runs the flow once on target — proves Fernet decryption still works end-to-end (the silent "API key required while UI shows green" failure mode documented in the langflow-image-migration skill). - During stress, a paced loop fires 9 additional flow executions spread evenly across the duration, while N users hammer cheap auth/list endpoints. Total: 10 OpenAI calls per matrix entry (1 smoke test + 9 paced), bounding cost. If the OPENAI_API_KEY secret is not set, the flow/variable/execution coverage is skipped with a warning; the migration test still runs on the empty stub witness so schema migration can be exercised without the cost. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 tasks
Stops creating the OPENAI_API_KEY Credential via POST /api/v1/variables/ after Langflow is already up — that path is brittle because the Simple Agent starter resolves its credential binding at startup, and a Variable created post-startup does not get picked up by component templates. Langflow has native support for auto-importing OPENAI_API_KEY from the process environment as a Credential-typed Global Variable at startup (reference: docs.langflow.org/configuration-global-variables). Wiring the secret to the source container via `-e OPENAI_API_KEY` produces a real Fernet-encrypted row in the DB, identical to what a user would get from the UI. The target container intentionally does NOT receive OPENAI_API_KEY: that way, the post-migration flow-execution check depends on the encrypted Variable surviving the schema migration (and the SECRET_KEY still being able to decrypt it), not on a re-import by the target process. This keeps the test rigorous: a regression that drops or mangles the Variable in transit fails loudly. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 tasks
…manual
Extends the manual migration workflow with a `docker-compose` value for
the `database` input, alongside `postgresql` and `sqlite`. The "both"
option is renamed to `all` to reflect that three scenarios are now
available.
Behind the scenes, a DEPLOY_MODE env var (set in Resolve runtime
configuration) drives the source/target start/stop/cleanup steps:
- DEPLOY_MODE=direct → inline `docker run` against a Postgres
container (or sqlite bind mount), same as before. Used for the
`postgresql` and `sqlite` matrix values.
- DEPLOY_MODE=compose → uses the OFFICIAL upstream docker-compose
file from langflow-ai/langflow/docker_example/docker-compose.yml
(fetched at runtime). Same source/target image inputs, but the
image is injected via a compose override + .env, and Postgres is
managed by the compose's named volume. Used for `docker-compose`.
API-level steps (user creation, witness flow, verify, execute, stress,
log scan) are identical across all three modes. Wait-healthy switched
from `docker logs | grep "Open Langflow"` to `curl /health_check` so it
works uniformly for both deployment modes.
Target intentionally clears OPENAI_API_KEY in compose mode too — the
post-migration flow execution depends on the migrated/encrypted
Variable row in the postgres volume, not on a re-import by the target
process. Same rigor principle as the direct mode.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 18, 2026
Generates /tmp/migration-summary.md at the end of every matrix entry (postgresql / sqlite / docker-compose), uploaded with the existing logs. The summary adapts to the matrix value: includes deploy mode, witness flow id, user count, OPENAI_API_KEY availability, and a checklist of what was verified end-to-end. Same pattern as #273 (migration-upgrade-with-flows) and #274 (migration-test, both jobs). The structure stays consistent across all migration workflows so the team can review runs uniformly. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new on-demand GitHub Actions workflow that performs a release-grade migration test between a source and target Langflow image. Beyond the existing daily migration check, it boots the source image to seed N users, an OpenAI Credential, and a witness flow; boots the target on the same DB (running Alembic migrations); then drives concurrent multi-user load against the migrated instance and scans the target log for the runtime DB/session error signatures from langflow-ai/langflow#13157. The workflow supports postgresql, sqlite, and docker-compose deploy modes.
Changes:
- New
workflow_dispatchworkflow with inputs for source/target images, database scenario, workers, concurrent users, and stress duration. - Source → target migration harness with witness flow, user/variable preservation checks, and Fernet-decrypt smoke test.
- Concurrent stress phase with paced flow executions and post-run log scan for known #13157 error patterns.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+19
to
+28
| database: | ||
| description: "Scenario(s) to test. `docker-compose` uses the official compose from langflow-ai/langflow." | ||
| required: true | ||
| type: choice | ||
| options: | ||
| - postgresql | ||
| - sqlite | ||
| - docker-compose | ||
| - all | ||
| default: "all" |
Comment on lines
+373
to
+386
| if [[ "$DEPLOY_MODE" == "compose" ]]; then | ||
| # Flip the image and clear OPENAI_API_KEY in compose .env. Target | ||
| # intentionally has NO OPENAI_API_KEY env so it depends entirely on | ||
| # the migrated/encrypted Variable row from the source phase. | ||
| sed -i 's|^LANGFLOW_IMAGE=.*|LANGFLOW_IMAGE=${{ inputs.target_image }}|' "$COMPOSE_DIR/.env" | ||
| sed -i 's|^OPENAI_API_KEY=.*|OPENAI_API_KEY=|' "$COMPOSE_DIR/.env" | ||
| (cd "$COMPOSE_DIR" && docker compose up -d langflow) | ||
| else | ||
| docker run -d --name lf-target \ | ||
| --network host \ | ||
| $MOUNT_FLAGS \ | ||
| --env-file /tmp/lf.env \ | ||
| "${{ inputs.target_image }}" | ||
| fi |
Comment on lines
+548
to
+558
| while IFS= read -r TOKEN; do | ||
| ( | ||
| while [[ $(date +%s) -lt $END_TIME ]]; do | ||
| curl -s -o /dev/null -H "Authorization: Bearer ${TOKEN}" \ | ||
| "${LF_URL}/api/v1/users/whoami" | ||
| curl -s -o /dev/null -H "Authorization: Bearer ${TOKEN}" \ | ||
| "${LF_URL}/api/v1/flows/?skip=0&limit=50" | ||
| done | ||
| ) & | ||
| done < /tmp/tokens.txt | ||
| wait |
There was a problem hiding this comment.
Added sleep 0.1 inside the inner stress loop (after the two curl calls) in commit Add sleep 0.1 throttle inside concurrent stress loop. This caps each user goroutine to ~10 RPS, preventing CPU saturation on 2-CPU runners while keeping the concurrent load signal meaningful.
| echo "| Deploy mode | \`${DEPLOY_MODE}\` |" | ||
| echo "| Source image | \`${{ inputs.source_image }}\` (\`${SOURCE_DIGEST}\`) |" | ||
| echo "| Target image | \`${{ inputs.target_image }}\` (\`${TARGET_DIGEST}\`) |" | ||
| echo "| LANGFLOW_WORKERS | \`${{ inputs.workers }}\` (forced to 1 if SQLite) |" |
Comment on lines
+432
to
+438
| PRESENT=$(curl -s -H "Authorization: Bearer ${ADMIN_TOKEN}" \ | ||
| "${LF_URL}/api/v1/users/?skip=0&limit=1000" \ | ||
| | jq '[.users[] | select(.username | startswith("testuser_"))] | length') | ||
| if [[ "$PRESENT" -ne "$EXPECTED" ]]; then | ||
| echo "::error::User count mismatch after migration: expected ${EXPECTED}, found ${PRESENT}" | ||
| exit 1 | ||
| fi |
Comment on lines
+193
to
+200
| cat > "$COMPOSE_DIR/.env" <<EOF | ||
| LANGFLOW_IMAGE=${{ inputs.source_image }} | ||
| LANGFLOW_SECRET_KEY=${LANGFLOW_SECRET_KEY} | ||
| OPENAI_API_KEY=${OPENAI_API_KEY} | ||
| LANGFLOW_SUPERUSER=${SUPERUSER} | ||
| LANGFLOW_SUPERUSER_PASSWORD=${SUPERUSER_PASSWORD} | ||
| LANGFLOW_WORKERS=${EFFECTIVE_WORKERS} | ||
| EOF |
| WITNESS_ID=$(curl -s -X POST "${LF_URL}/api/v1/flows/" \ | ||
| -H "Authorization: Bearer ${ADMIN_TOKEN}" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"name":"migration-witness-manual","description":"Empty stub witness","data":{"nodes":[],"edges":[]}}' \ |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On-demand migration workflow designed for release-grade validation. Unlike
migration-test.yml(which runs daily and only verifies that the schema migration completes), this one also exercises the runtime conditions that surface DB session bugs — directly inspired by langflow-ai/langflow#13157.How it works
For each selected database backend (postgresql, sqlite, or both):
Inputs
Design decisions
Error patterns scanned in the target log
From the issue body of #13157:
Test plan
🤖 Generated with Claude Code