Skip to content

Commit edd3075

Browse files
Merge pull request #1161 from oriontech-me/ci/issue-1092-file-watcher-lfx-coverage
ci(file-watcher): watch src/lfx, and fail loudly on a path that no longer exists (#1092)
2 parents d884a8c + d1e91a7 commit edd3075

7 files changed

Lines changed: 1284 additions & 165 deletions

File tree

.claude/skills/langflow-e2e-infra/references/infra-map.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,9 +13,9 @@ before acting; this list drifts.
1313
| `daily-stable.yml` | **Active stable workflow.** Weekdays 05:00 BRT, `@stable` only; opens `daily-failure` issue; appends `reports/daily-history.jsonl` | consumed by `langflow-e2e-triage` |
1414
| `weekly-stable.yml` | **Disabled** fallback (superseded by daily-stable); writes `reports/weekly-history.jsonl` | frozen history |
1515
| `pr-validation.yml` | Every PR: `tsc --noEmit` + ESLint + QA-CHECKLIST guard + impacted-specs gate | `#741`, `#873`, `#892` |
16-
| `adaptive-impacted.yml` | Runs the impacted-tests subset for a PR | `scripts/impacted-tests.ts`; `CONTRIBUTING.md` → Adaptive impacted-tests |
16+
| `adaptive-impacted.yml` | Runs the impacted-tests subset for a PR. **`disabled_manually`** | `scripts/impacted-tests.ts`; `CONTRIBUTING.md` → Adaptive impacted-tests |
1717
| `manual.yml` | Parameterized manual run (Docker tag / URL, suite, grep) | use to dry-run a workflow-adjacent change |
18-
| `file-watcher.yml` | Detects upstream Langflow changes in critical paths; opens revalidation issue | `CONTRIBUTING.md` → Monitored areas |
18+
| `file-watcher.yml` | Detects upstream Langflow changes in monitored paths; opens revalidation issue. **`disabled_manually` in Actions + no cron (9da85fa) ⇒ no run history at all**; a dispatch 422s | `scripts/watch-upstream-areas.mjs` (area table + `lfx` decision record + fail-closed guard, `#1092`) |
1919
| `triage-dispatch.yml` | Automates daily-failure triage dispatch behind an approval gate | `#785/#786/#787`, `#819` |
2020
| `update-coverage-summary.yml` | Regenerates QA-CHECKLIST generated blocks on merge to `main` | `scripts/coverage-summary.ts`, `stable-tests.ts` |
2121
| `migration-test.yml` / `migration-fresh-install.yml` / `migration-upgrade-with-flows.yml` | Langflow version-migration checks (latest → nightly) | `migration-test` label |

.github/workflows/file-watcher.yml

Lines changed: 60 additions & 142 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,27 @@
11
name: Langflow Source File Watcher
22

3+
# Opens a revalidation issue when upstream Langflow touches a path one of our
4+
# areas depends on. The monitored-area table, the existence guard and the change
5+
# sweep all live in `scripts/watch-upstream-areas.mjs` (issue #1092) — the table
6+
# used to be inline bash here, where `git log -- <gone-path>` made a dead path
7+
# indistinguishable from a quiet one, and `src/lfx/` (the package Langflow keeps
8+
# moving backend behavior into) was watched by no area at all.
9+
#
10+
# Note this workflow is `workflow_dispatch` only — the schedule was disabled in
11+
# 9da85fa. `since` therefore defaults to 24h but exists so a dispatch after a
12+
# quiet stretch can cover the real gap.
13+
314
on:
4-
workflow_dispatch: {}
15+
workflow_dispatch:
16+
inputs:
17+
since:
18+
description: '"N hours|days|weeks|months ago", "yesterday", or an ISO date. Validated — a typo is rejected, not guessed at by git approxidate'
19+
required: false
20+
default: '24 hours ago'
21+
22+
permissions:
23+
contents: read
24+
issues: write
525

626
jobs:
727
watch-langflow-changes:
@@ -20,155 +40,53 @@ jobs:
2040
path: langflow-upstream
2141
fetch-depth: 0 # full history needed for git log
2242

23-
- name: Detect changes in monitored paths (last 24h)
43+
# Fail-closed: an area whose paths no longer exist cannot be evaluated, and
44+
# must not read as clean (same rule as the daily's runguard, #1012). This
45+
# also fails when upstream adds an `lfx` subtree nothing classifies, so the
46+
# next package split forces a decision instead of widening the blind spot.
47+
#
48+
# `continue-on-error` deliberately: upstream renames constantly, and one
49+
# dead path must not suppress the report for the other 12 areas. The
50+
# annotations land here, the report is still produced, and the job is failed
51+
# at the end by "Enforce the path guard" — coverage first, strictness kept
52+
# (the trade-off `Collect models` makes on the daily, #980/#570).
53+
- name: Guard the monitored paths against the checkout
54+
id: guard
55+
continue-on-error: true
56+
run: node scripts/watch-upstream-areas.mjs --mode=check --root langflow-upstream
57+
58+
- name: Detect changes in monitored paths
2459
id: detect
25-
working-directory: langflow-upstream
26-
run: |
27-
SINCE="24 hours ago"
28-
29-
declare -A AREA_PATHS
30-
declare -A AREA_TAGS
31-
declare -A AREA_CHECKLIST
32-
33-
# ── ROUTES & FEATURE FLAGS (highest priority — can break everything) ──
34-
AREA_PATHS["Routes & Feature Flags"]="src/frontend/src/routes.tsx src/frontend/src/customization/feature-flags.ts src/frontend/src/customization/config-constants.ts"
35-
AREA_TAGS["Routes & Feature Flags"]="@release @api @workspace @components"
36-
AREA_CHECKLIST["Routes & Feature Flags"]="ALL AREAS — route or feature flag changes can break any test"
37-
38-
# ── AUTHENTICATION ──
39-
AREA_PATHS["Authentication"]="src/backend/base/langflow/api/v1/login.py src/backend/base/langflow/api/v1/users.py src/backend/base/langflow/api/v1/api_key.py src/backend/base/langflow/services/auth/ src/frontend/src/pages/LoginPage/ src/frontend/src/pages/SignUpPage/ src/frontend/src/components/authorization/ src/frontend/src/stores/authStore.ts"
40-
AREA_TAGS["Authentication"]="@release @api"
41-
AREA_CHECKLIST["Authentication"]="AREA 1 — Authentication and User Management"
42-
43-
# ── FLOW CRUD & CANVAS ──
44-
AREA_PATHS["Flow CRUD & Canvas"]="src/backend/base/langflow/api/v1/flows.py src/backend/base/langflow/api/v1/folders.py src/frontend/src/pages/FlowPage/ src/frontend/src/stores/flowStore.ts src/frontend/src/stores/flowsManagerStore.ts src/frontend/src/constants/flow_constants.tsx"
45-
AREA_TAGS["Flow CRUD & Canvas"]="@workspace @release"
46-
AREA_CHECKLIST["Flow CRUD & Canvas"]="AREA 2 — Flow CRUD | AREA 3 — Folders | AREA 4 — Canvas Editor"
47-
48-
# ── FLOW EXECUTION ──
49-
AREA_PATHS["Flow Execution"]="src/backend/base/langflow/api/v1/endpoints.py src/backend/base/langflow/processing/ src/backend/base/langflow/api/v1/chat.py src/backend/base/langflow/services/chat/"
50-
AREA_TAGS["Flow Execution"]="@release @api"
51-
AREA_CHECKLIST["Flow Execution"]="AREA 10.3 — Flow Execution via API | AREA 4.10 — Run and Stop"
52-
53-
# ── MODEL PROVIDERS & LLM ──
54-
AREA_PATHS["Model Providers & LLM"]="src/frontend/src/pages/SettingsPage/pages/ModelProvidersPage/ src/frontend/src/modals/modelProviderModal/ src/frontend/src/constants/providerConstants.ts src/frontend/src/components/common/modelProviderCountComponent/ src/backend/base/langflow/api/v1/models.py src/backend/base/langflow/api/v1/model_options.py"
55-
AREA_TAGS["Model Providers & LLM"]="@components @api"
56-
AREA_CHECKLIST["Model Providers & LLM"]="AREA 8 — LLM Integrations and Model Providers | AREA 7 — Templates (all integration tests)"
57-
58-
# ── AGENTS & AGENTIC FLOWS ──
59-
AREA_PATHS["Agents & Agentic Flows"]="src/backend/base/langflow/agentic/ src/backend/base/langflow/base/agents/ src/frontend/src/pages/MainPage/"
60-
AREA_TAGS["Agents & Agentic Flows"]="@components @release"
61-
AREA_CHECKLIST["Agents & Agentic Flows"]="AREA 9.5 — Agent Component | AREA 7.4 — Agent Templates"
62-
63-
# ── PLAYGROUND & CHAT UI ──
64-
AREA_PATHS["Playground & Chat"]="src/frontend/src/pages/Playground/ src/frontend/src/components/core/playgroundComponent/ src/frontend/src/stores/playgroundStore.ts src/frontend/src/stores/messagesStore.ts src/backend/base/langflow/api/v1/chat.py"
65-
AREA_TAGS["Playground & Chat"]="@workspace @release"
66-
AREA_CHECKLIST["Playground & Chat"]="AREA 6 — Playground"
67-
68-
# ── GLOBAL VARIABLES & SETTINGS ──
69-
AREA_PATHS["Settings & Global Variables"]="src/frontend/src/pages/SettingsPage/ src/backend/base/langflow/api/v1/variable.py src/frontend/src/stores/globalVariablesStore/ src/backend/base/langflow/settings.py"
70-
AREA_TAGS["Settings & Global Variables"]="@api @release"
71-
AREA_CHECKLIST["Settings & Global Variables"]="AREA 8.4 — Global Variables | AREA 13 — Settings"
72-
73-
# ── MCP SERVER ──
74-
AREA_PATHS["MCP Server"]="src/frontend/src/pages/SettingsPage/pages/MCPServersPage/ src/frontend/src/modals/addMcpServerModal/ src/backend/base/langflow/api/v1/mcp.py src/backend/base/langflow/api/v1/mcp_projects.py src/backend/base/langflow/agentic/mcp/"
75-
AREA_TAGS["MCP Server"]="@components"
76-
AREA_CHECKLIST["MCP Server"]="AREA 11 — MCP Server"
77-
78-
# ── TRACING & MONITORING ──
79-
AREA_PATHS["Tracing & Monitoring"]="src/backend/base/langflow/api/v1/traces.py src/backend/base/langflow/api/v1/monitor.py src/backend/base/langflow/services/tracing/"
80-
AREA_TAGS["Tracing & Monitoring"]="@api"
81-
AREA_CHECKLIST["Tracing & Monitoring"]="AREA 12 — Observability and Monitoring"
82-
83-
# ── DATABASE MODELS ──
84-
AREA_PATHS["Database Models"]="src/backend/base/langflow/services/database/models/ src/backend/base/langflow/alembic/"
85-
AREA_TAGS["Database Models"]="@database @release"
86-
AREA_CHECKLIST["Database Models"]="AREA 1, 2, 6, 8 — any area with persisted state"
87-
88-
# ── COMPONENT INPUT TYPES ──
89-
AREA_PATHS["Component Input Types"]="src/frontend/src/components/core/parameterRenderComponent/ src/frontend/src/CustomNodes/ src/backend/base/langflow/inputs/ src/backend/base/langflow/field_typing/"
90-
AREA_TAGS["Component Input Types"]="@components"
91-
AREA_CHECKLIST["Component Input Types"]="AREA 5 — Component Configuration | AREA 4 — Canvas Editor"
92-
93-
# ── FILE UPLOAD ──
94-
AREA_PATHS["File Upload"]="src/backend/base/langflow/api/v1/files.py src/backend/base/langflow/services/storage/ src/frontend/src/pages/MainPage/pages/filesPage/"
95-
AREA_TAGS["File Upload"]="@components @api"
96-
AREA_CHECKLIST["File Upload"]="AREA 9.7 — File Upload Component"
97-
98-
# ────────────────────────────────────────────────────────────────────
99-
CHANGED_AREAS=""
100-
CHANGE_DETAILS=""
101-
102-
for AREA in "${!AREA_PATHS[@]}"; do
103-
PATH_LIST="${AREA_PATHS[$AREA]}"
104-
COMMITS=$(git log --since="$SINCE" --oneline -- $PATH_LIST 2>/dev/null | head -5)
105-
if [ -n "$COMMITS" ]; then
106-
TAGS="${AREA_TAGS[$AREA]}"
107-
CHECKLIST="${AREA_CHECKLIST[$AREA]}"
108-
CHANGED_AREAS="$CHANGED_AREAS\n- **$AREA** | tags: \`$TAGS\` | checklist: $CHECKLIST"
109-
CHANGE_DETAILS="$CHANGE_DETAILS\n\n### $AREA\n\`\`\`\n$COMMITS\n\`\`\`"
110-
fi
111-
done
112-
113-
if [ -n "$CHANGED_AREAS" ]; then
114-
echo "has_changes=true" >> $GITHUB_OUTPUT
115-
{
116-
echo "areas<<EOF"
117-
echo -e "$CHANGED_AREAS"
118-
echo "EOF"
119-
} >> $GITHUB_OUTPUT
120-
{
121-
echo "details<<EOF"
122-
echo -e "$CHANGE_DETAILS"
123-
echo "EOF"
124-
} >> $GITHUB_OUTPUT
125-
else
126-
echo "has_changes=false" >> $GITHUB_OUTPUT
127-
fi
60+
env:
61+
SINCE: ${{ inputs.since || '24 hours ago' }}
62+
run: node scripts/watch-upstream-areas.mjs --mode=detect --root langflow-upstream --since "$SINCE"
12863

12964
- name: Create review issue if changes detected
13065
if: steps.detect.outputs.has_changes == 'true'
13166
uses: actions/github-script@v9
67+
env:
68+
# Read through the environment, never interpolated into the script body:
69+
# the body embeds upstream commit subjects, which are untrusted input.
70+
# Rendering lives in the script so `npm run test:scripts` covers it.
71+
ISSUE_TITLE: ${{ steps.detect.outputs.title }}
72+
ISSUE_BODY: ${{ steps.detect.outputs.body }}
13273
with:
13374
script: |
134-
const today = new Date().toISOString().split('T')[0];
135-
const areas = `${{ steps.detect.outputs.areas }}`;
136-
const details = `${{ steps.detect.outputs.details }}`;
137-
138-
const body = [
139-
'## Langflow source changes detected — test review required',
140-
'',
141-
`**Date:** ${today}`,
142-
`**Source:** [langflow-ai/langflow](https://github.com/langflow-ai/langflow)`,
143-
'',
144-
'### Changed areas, affected tags and checklist sections',
145-
'',
146-
'| Area | Run these tests | Checklist |',
147-
'|---|---|---|',
148-
...areas.split('\n').filter(l => l.startsWith('- **')).map(line => {
149-
const match = line.match(/\*\*(.+?)\*\* \| tags: `(.+?)` \| checklist: (.+)/);
150-
if (match) return `| ${match[1]} | \`npx playwright test --grep "${match[2].split(' ').join('|')}"\` | ${match[3]} |`;
151-
return line;
152-
}),
153-
'',
154-
'### Commits (last 24h)',
155-
details,
156-
'',
157-
'### Action required',
158-
'1. Review the commits listed above',
159-
'2. Run the tests indicated for each changed area',
160-
'3. Follow the validation guide in README.md → "Como validar um teste existente"',
161-
'4. Update or fix stale tests if behavior changed',
162-
'5. Update `REGRESSION_CHECKLIST.md` if new scenarios were introduced',
163-
'6. Close this issue when done',
164-
'',
165-
'/cc @lice-reis',
166-
].join('\n');
167-
16875
await github.rest.issues.create({
16976
owner: context.repo.owner,
17077
repo: context.repo.repo,
171-
title: `[Test Review] Langflow source changed on ${today} — validate affected tests`,
172-
body,
173-
labels: ['needs-review', 'langflow-upstream-change'],
78+
title: process.env.ISSUE_TITLE,
79+
body: process.env.ISSUE_BODY,
80+
// Both labels exist in the repo. The previous pair
81+
// (needs-review / langflow-upstream-change) did not, so the first
82+
// successful dispatch would have auto-created two labels nobody
83+
// decided on.
84+
labels: ['needs-triage', 'automated'],
17485
});
86+
87+
# The guard's verdict decides the job, but only after the report exists.
88+
- name: Enforce the path guard
89+
if: always() && steps.guard.outcome == 'failure'
90+
run: |
91+
echo "::error::the monitored-path guard failed — see the 'Guard the monitored paths' step." >&2
92+
exit 1

0 commit comments

Comments
 (0)