QA-CHECKLIST.md

Langflow — Regression Test Checklist

Repository: C:/QAx/langflow-playwright/langflow-e2e Tests: tests/tests-automations/regression/ Config: playwright.config.ts Last updated: 2026-08-08

---

How to use this checklist

• [x] → automated and validated — only assigned when the underlying Playwright test carries the @stable tag
• [-] → automated, needs validation (test exists but is not yet @stable, or the entry refers to a Page/Helper rather than a test)
• [ ] (empty) → needs automation
• [~] → partially covered
• [!] → covered but flaky / unstable

---

---

PART I — PAGES & HELPERS

---

Pages

• [-] SimpleAgentTemplatePage — loads Simple Agent template with configurable provider and model → pages/SimpleAgentTemplatePage.ts
• [-] SettingsPage — navigation to the settings page via user menu → pages/SettingsPage.ts
• [-] Component sidebar — component navigation bar with searchable parameter support → covered by ui-ux/sidebar-search-and-filter.spec.ts, ui-ux/keyboardComponentSearch.spec.ts, ui-ux/sidebar-add-component.spec.ts (#820: no dedicated nav spec needed — exercised across the sidebar suite; #937 consolidated sidebar-provider-count / sidebar-category-filter / sidebar-filter-by-category into sidebar-search-and-filter)
• [-] Model Provider — navigation to the model provider management tab → covered by ui-ux/settings-navigation.spec.ts ("Settings Model Providers section loads") (#820)
• [-] API Keys — navigation to the API keys / global variables tab → covered by ui-ux/userSettings.spec.ts (API Keys), ui-ux/global-variable-edit.spec.ts + ui-ux/global-variables-crud.spec.ts (global variables) (#820)
• [-] Templates — navigation to the template selection tab (Starter Projects) → covered by core-functionality/templates/starter-projects.spec.ts, flow-functionality/create-flow-from-template.spec.ts (#820)
• Import Flow — navigation to import a flow via JSON → flow-functionality/export-import-flow.spec.ts (also flow-functionality/import-invalid-json.spec.ts; see §12.4) (#820)
• Delete Flow — navigation to delete a flow → ui-ux/actionsMainPage-shard-1.spec.ts ("select and delete a flow"); bulk via core-functionality/project-management/bulk-actions.spec.ts (#820)
• [-] MCP Config — navigation to configure MCP Server → covered by mcp/server/mcp-server-tab.spec.ts, mcp/server/mcp-server.spec.ts, core-components/configure-mcp-and-custom-component.spec.ts (#820)

---

Helpers

Provider Setup

• [-] OpenAI Provider Setup → helpers/provider-setup/setup-openai.ts
• [-] Anthropic Provider Setup → helpers/provider-setup/setup-anthropic.ts
• [-] Google Generative AI Provider Setup → helpers/provider-setup/setup-google.ts
• [-] Provider Map (providerSetupMap) — central registration point → helpers/provider-setup/index.ts
• [-] Provider validation via API (credit, valid key) → helpers/provider-setup/collect-models.ts
• [-] Provider build-axis probe (registry key present + component instantiates) → helpers/provider-setup/probe-component-buildable.ts
• [-] Collection of available models via UI (Settings → Model Providers) → helpers/provider-setup/collect-models.ts
• [-] providers.json — status of each provider (active/inactive + reason) → data/providers.json
• [-] models.json — list of models per provider → data/models.json

Flows

• [-] Load Simple Agent with variable provider and model → pages/SimpleAgentTemplatePage.ts
• [-] Load Simple Agent with OpenAI (wrapper) → helpers/flows/load-simple-agent-with-openai.ts

To implement

• [-] Configure an MCP → helpers/mcp/configure-mcp-server.ts
• [-] Configure a Custom Component → helpers/flows/configure-custom-component.ts
• Delete a component → helpers/flows/delete-component.ts
• Run a flow → helpers/flows/run-flow.ts
• [-] Pause a flow → flow-functionality/stop-building.spec.ts
• Send a chat input → flow-functionality/flow-execution-canvas.spec.ts
• Verify the chat output → flow-functionality/flow-execution-canvas.spec.ts

---

---

PART II — TEST AUTOMATION COVERAGE

Organized according to tests/tests-automations/regression/

---

api/ — REST API

api/flows/ — REST API

1.1 Health Check

• GET /health_check → status 200, db ok → api-health-check.spec.ts
• GET /api/v1/version → returns version, main_version, package → api-version.spec.ts

1.2 Flow CRUD via API

• POST /api/v1/flows/ → creates flow, returns ID → api/flows/api-flows-crud.spec.ts
• GET /api/v1/flows/ → lists user flows → api/flows/api-flows-crud.spec.ts
• GET /api/v1/flows/{id} → returns flow by ID → api/flows/api-flows-crud.spec.ts
• PATCH /api/v1/flows/{id} → updates name/description → api/flows/api-flows-crud.spec.ts
• DELETE /api/v1/flows/{id} → removes flow, returns 200 → api/flows/api-flows-crud.spec.ts
• GET /api/v1/flows/{id} after DELETE → should return 404 → api/flows/api-flows-crud.spec.ts

1.3 Flow Execution via API

• POST /api/v1/run/{flow_id} with input_value → returns response → api/flows/api-run-flow.spec.ts
• POST with tweaks → parameters override flow configuration → api/flows/api-run-with-tweaks.spec.ts
• POST with custom session_id → api/flows/api-run-flow.spec.ts
• POST with custom session_id persists messages retrievable via GET /api/v1/monitor/messages?session_id → api/flows/api-run-flow.spec.ts
• POST with input_type: "chat" and output_type: "chat" → api/flows/api-run-flow.spec.ts
• POST with invalid API key → returns 401/403 → api-invalid-key.spec.ts
• POST to non-existent flow → returns 404 → api/flows/api-run-flow.spec.ts

1.3.1 Folder (Project) CRUD via API

• POST /api/v1/projects/ → creates folder, returns id and name → api/flows/api-folders-crud.spec.ts
• GET /api/v1/projects/ → lists folders including the created one → api/flows/api-folders-crud.spec.ts
• [!] DELETE /api/v1/projects/{id} → returns 204 and the folder leaves the listing — quarantined (#965): under concurrent writes the endpoint answers 500 (sqlite3.OperationalError: database is locked) and the folder survives; measured 44% of deletes on 1.12.0.dev7 vs 6% on stable 1.10.3 at the same 2-client contention. Product defect filed as LE-2020; the 204 assertion is unchanged. The same ticket also covers PATCH /api/v1/flows/{id} (§12.5, #932) — one root cause, two endpoints → api/flows/api-folders-crud.spec.ts

1.4 Components via API

• GET /api/v1/all → lists all available components → api/flows/api-custom-component-creation.spec.ts
• POST /api/v1/custom_component → creates custom component → api/flows/api-custom-component-creation.spec.ts

1.5 Messages and Monitoring via API

• GET /api/v1/monitor/messages → returns 200 with array → api/flows/api-monitor-messages.spec.ts
• GET with session_id filter returns only messages from that session → api/flows/api-monitor-messages.spec.ts

1.6 Integration Code Generation

• Generate curl for API execution → flow-functionality/curlApiGeneration.spec.ts
• Generate Python code for integration → flow-functionality/pythonApiGeneration.spec.ts
• API access modal → flow-functionality/api-access-modal-regression.spec.ts

1.7 API Key Serialization & Expiry (PR #13471)

• GET /api/v1/api_key/ serializes created_at/expires_at as UTC ISO with +00:00 offset and no microseconds; null expires_at/last_used_at stay null → ui-ux/api-keys-timezone-display.spec.ts
• Expired API key is rejected on POST /api/v1/run/{id} with 403; valid key accepted with 200 → api/flows/api-key-expiry-enforcement.spec.ts
• Expiry boundary is evaluated in UTC, not shifted by viewer offset (±30 min UTC keys resolve correctly) → api/flows/api-key-expiry-enforcement.spec.ts

---

core-components/ — Component Configuration + Core Components

2. Component Configuration

2.1 Parameters Panel

• [-] Open component advanced options
• Edit text field (input) → core-components/parameters-panel-field-types.spec.ts
• Edit dropdown → core-components/parameters-panel-field-types.spec.ts
• Edit text area (textarea) → core-components/parameters-panel-field-types.spec.ts
• Edit code field → core-components/parameters-panel-field-types.spec.ts
• Edit float field → core-components/parameters-panel-field-types.spec.ts
• Edit int field → core-components/parameters-panel-field-types.spec.ts
• Edit toggle field → core-components/parameters-panel-field-types.spec.ts
• Edit key-pair list → core-components/parameters-panel-field-types.spec.ts
• Edit input list → core-components/parameters-panel-field-types.spec.ts
• Edit table input → core-components/parameters-panel-field-types.spec.ts
• Edit slider → core-components/parameters-panel-field-types.spec.ts
• Edit tab component → core-components/parameters-panel-field-types.spec.ts
• [-] Visibility toggle of a connected input is disabled (tooltip "Cannot change visibility of connected handles") and re-enables once the edge is deleted → flow-functionality/general-bugs-hidden-input-edges.spec.ts
• Two nodes on the canvas exposing the same field name render distinct DOM ids, while data-testid stays unscoped so both nodes remain selectable (LE-2037 / langflow#14312) → core-components/duplicate-dom-ids-regression.spec.ts

2.2 Tool Mode

• Enable Tool Mode on a component → core-components/tool-mode.spec.ts
• Group components in Tool Mode → core-components/tool-mode-group.spec.ts
• Edit tools (slug, description, requires-approval persistence) → core-components/edit-tools.spec.ts

2.3 Component Updates

• Outdated component notification → core-components/outdated-component-notification.spec.ts
• Update component action → core-components/update-component-action.spec.ts
• Update with breaking change — should alert user → core-components/component-breaking-change-alert.spec.ts
• Legacy component visible via configuration → core-components/legacy-components-toggle-regression.spec.ts
• Beta component visible via configuration → core-components/beta-components-toggle-regression.spec.ts
• Re-saving code removes handles from previously-toggled advanced fields → core-components/general-bugs-delete-handle-advanced-input.spec.ts

2.4 Code Editing

• Edit Python code of custom component — Check & Save clears the pulse-pink indicator → core-components/customComponentAdd.spec.ts
• Full custom component → core-components/full-custom-component.spec.ts
• [-] configureCustomComponent helper compiles code into a node with its declared interface → core-components/configure-mcp-and-custom-component.spec.ts

---

3. Core Components

3.1 Chat Input / Output

• ChatInput renders on canvas with Message output handle and Input Text field → core-components/chat-input-output-component-regression.spec.ts
• ChatOutput renders on canvas with Inputs handle and run button → core-components/chat-input-output-component-regression.spec.ts
• ChatInput → ChatOutput connection accepted (Message ↔ Message) → core-components/chat-input-output-component-regression.spec.ts
• Input Text propagates from ChatInput to ChatOutput on run → core-components/chat-input-output-component-regression.spec.ts
• Sender name override is reflected in the Playground chat message → core-components/chat-input-output-component-regression.spec.ts
• Default sender_name is "User" on input and "AI" on output → core-components/chat-input-output-component-regression.spec.ts
• Toggling showfiles exposes the Files inspector field on Chat Input → core-components/chat-input-files-field-regression.spec.ts
• Uploading a file via the Chat Input inspector populates the Files field → core-components/chat-input-files-field-regression.spec.ts
• Inspector-attached file is rendered in the Playground after running ChatInput → ChatOutput → core-components/chat-input-files-field-regression.spec.ts
• Dismiss button on the Files field clears the value → core-components/chat-input-files-field-regression.spec.ts
• Chat Input is a singleton — adding one removes both the Chat Input and Webhook + buttons from the sidebar (mutual exclusion) → core-components/singleton-components.spec.ts
• A value typed on a node (Chat Input as the host) is persisted by the debounced autosave and rehydrated after leaving and re-entering the flow — four consecutive edits, each gated on the server before the exit → core-components/general-bugs-save-changes-on-node.spec.ts
• Chat Input cannot be duplicated (Cmd/Ctrl+D) or copy/pasted (Cmd/Ctrl+C+V) — blocked with the "components were not pasted" toast → core-components/singleton-components.spec.ts
• [-] File on the advanced files field can be removed and re-uploaded; after running, the image and the user message render in the Playground → flow-functionality/general-bugs-shard-3836.spec.ts

3.2 Prompt Template

• Prompt Template renders on canvas with output handle → core-components/prompt-template-component-regression.spec.ts
• Variables in curly braces generate dynamic input handles → core-components/prompt-template-component-regression.spec.ts
• Removing a variable removes its input handle → core-components/prompt-template-component-regression.spec.ts
• Replacing a variable updates handles accordingly → core-components/prompt-template-component-regression.spec.ts
• Clearing the template removes all dynamic handles → core-components/prompt-template-component-regression.spec.ts
• Modal edits persist in UI and in saved flow → core-components/prompt-template-component-regression.spec.ts
• use_double_brackets toggle is exposed in the InspectionPanel with its upstream display name → core-components/prompt-template-double-brackets-regression.spec.ts
• Default toggle state is OFF; f-string mode extracts {var} and treats {{var}} as literal → core-components/prompt-template-double-brackets-regression.spec.ts
• Enabling toggle switches parser to mustache mode; {{var}} creates handle and {var} is ignored → core-components/prompt-template-double-brackets-regression.spec.ts
• Disabling toggle reverts to f-string mode and variables are re-extracted under the new parser → core-components/prompt-template-double-brackets-regression.spec.ts
• use_double_brackets value persists in the autosaved flow → core-components/prompt-template-double-brackets-regression.spec.ts
• f-string parser rejects {var.attr} (dot notation) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser rejects {var name} (space inside identifier) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser rejects {var,name} (comma inside identifier) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser rejects {1var} (leading digit) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser accepts {} (empty braces) silently — no error, no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser deduplicates repeated variables — {name} and {name} yields exactly one handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• mustache parser rejects {{ var }} (spaces inside braces) with an error toast and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts
• mustache parser rejects {{var.attr}} (dot notation) with an error toast and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts
• mustache parser rejects {{#section}}{{/section}} with the complex-syntax message and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts
• mustache parser rejects {{{var}}} (triple braces) with the complex-syntax message and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts

3.3 API Request (HTTP)

• Renders on canvas with URL and API Response handles → core-components/api-request-component-regression.spec.ts
• Inspector fields accept URL and HTTP method values → core-components/api-request-component-regression.spec.ts
• Execute GET request and verify 200 status and output structure → core-components/api-request-component-regression.spec.ts
• Execute POST request and verify POST verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Execute PUT request and verify PUT verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Execute PATCH request and verify PATCH verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Execute DELETE request and verify DELETE verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Non-2xx HTTP response (404) propagated as status_code without crash → core-components/api-request-component-regression.spec.ts
• Query parameters embedded in URL are sent and echoed in response → core-components/api-request-component-regression.spec.ts
• Invalid URL error shows notification with descriptive error message → core-components/api-request-component-regression.spec.ts
• Headers table accepts key + value cell entries via inspector → core-components/api-request-component-regression.spec.ts
• cURL tab switches mode and exposes the cURL input field → core-components/api-request-component-regression.spec.ts
• cURL parser auto-fills URL field and executes the GET, returning 200 → core-components/api-request-component-regression.spec.ts
• Body table accepts key + value cell entries when method is POST (body field is advanced=True and hidden by inspector while method is GET) → core-components/api-request-component-regression.spec.ts
• Flow state (URL, method, headers row) persists in database after autosave and rehydrates on reload → core-components/api-request-component-regression.spec.ts
• include_httpx_metadata=true adds the outgoing request headers as a top-level headers key in the output Data (advanced field, added to the node body via the inspector) → api/flows/api-component-regression.spec.ts
• Request timeout shorter than the endpoint's delay returns status_code 500 with an error field instead of raising (advanced field, added to the node body via the inspector) → api/flows/api-component-regression.spec.ts

3.4 Webhook

• POST aceita JSON e text/plain retornando 202 com status: "in progress" → core-components/webhook-component-regression.spec.ts
• Flow salvo no banco contém o nó Webhook com endpoint="BACKEND_URL" → core-components/webhook-component-regression.spec.ts (quarentena de 2½ meses liberada em #990: o teste lia o flow via page.evaluate(fetch) e tropeçava no defeito do interceptor de window.fetch do frontend; agora lê via request.get autenticado)
• Campo cURL no inspector mostra URL válida com flow ID e flags corretas (-X POST, Content-Type, -d) → core-components/webhook-component-regression.spec.ts
• Data field vazia retorna objeto Data vazio {} ao executar → core-components/webhook-component-regression.spec.ts
• Campo endpoint (str_endpoint) renderiza a URL real do webhook → core-components/webhook-component-regression.spec.ts
• Botão de cópia copia a URL correta para o clipboard e exibe toast "Endpoint URL copied" → core-components/webhook-component-regression.spec.ts
• POST para flow inexistente retorna 404 → core-components/webhook-component-regression.spec.ts
• GET /api/v1/monitor/messages retorna 200 com array → core-components/webhook-component-regression.spec.ts
• Payload JSON recebido é propagado corretamente como saída Data do componente → core-components/webhook-component-regression.spec.ts
• Payload inválido (não-JSON) é encapsulado em {"payload": "..."} na saída → core-components/webhook-component-regression.spec.ts
• Webhook is a singleton — adding one removes both the Webhook and Chat Input + buttons from the sidebar (mutual exclusion) → core-components/singleton-components.spec.ts
• Webhook cannot be duplicated (Cmd/Ctrl+D) or copy/pasted (Cmd/Ctrl+C+V) — blocked with the "components were not pasted" toast → core-components/singleton-components.spec.ts
• [-] Generated cURL includes the x-api-key header when webhook auth is enabled (mocked GET /api/v1/config, auto-login off) → core-components/general-bugs-component-webhook-api-key-display.spec.ts
• [-] Generated cURL omits x-api-key when webhook auth is disabled → core-components/general-bugs-component-webhook-api-key-display.spec.ts

3.5 Agent (Component)

• Agent component renders on canvas with title, handles and default fields → core-components/agent-component-regression.spec.ts
• System prompt accepts input and persists across flow reload → core-components/agent-component-regression.spec.ts
• Model dropdown exposes manage-model-providers and lists configured models → core-components/agent-component-regression.spec.ts
• Selecting a different-provider model swaps the canvas provider icon → core-components/agent-component-regression.spec.ts

3.6 Loop Component

• Loop component renders on canvas with title and run button → core-components/loop-component-regression.spec.ts
• Correct handles: inputs-left, item-left, item-right, done-right → core-components/loop-component-regression.spec.ts
• Output inspection buttons present for item and done → core-components/loop-component-regression.spec.ts
• Run without connections shows "Flow build failed" notification without crash → core-components/loop-component-regression.spec.ts
• Loop iterates over 2 ArXiv articles (Research Translation Loop template) and aggregates response in Playground → core-components/loop-component-regression.spec.ts
• Loop stops when exit condition is met → core-components/loop-component-regression.spec.ts

3.7 Nested / Grouping

• Nested component → core-components/nested-grouping-regression.spec.ts
• Enter and exit grouped component → core-components/nested-grouping-regression.spec.ts

3.8 If-Else Component

• operator=equals: matching input routes through True branch (False branch stays inactive) → core-components/if-else-component-regression.spec.ts
• operator=equals: non-matching input routes through False branch (True branch stays inactive) → core-components/if-else-component-regression.spec.ts
• operator=contains substring routing → core-components/if-else-component-regression.spec.ts
• operator=regex valid pattern routing → core-components/if-else-component-regression.spec.ts
• operator=regex hides case_sensitive field via update_build_config → core-components/if-else-component-regression.spec.ts
• case_sensitive ON (default) treats mixed case as no-match → core-components/if-else-component-regression.spec.ts
• case_sensitive OFF treats mixed case as a match → core-components/if-else-component-regression.spec.ts
• operator=greater than numeric routing → core-components/if-else-component-regression.spec.ts
• Other numeric operators (less than, less than or equal, greater than or equal) — share the same float(...) cast as greater than → core-components/if-else-component-regression.spec.ts
• max_iterations + default_route cycle break (not implementable as a standalone If-Else feedback loop on 1.12.x — Langflow forms graph cycles only via loop-aware target handles (from_loop_target_handle, target_handle.type is None) that LoopComponent ports provide; a feedback edge into the router's regular match_text field-input persists in the flow JSON but does not make the graph iterate (is_cyclic stays false, router runs once). conditional_router.py's cycle-break only fires when the router already sits inside a Loop-created cycle. Confirmed live on 1.12.0.dev3; product finding filed upstream; #891, follow-up of #822)

3.9 Human Input (HITL, 1.11.0)

• Human Input node config: default Approve/Reject branch handles, custom User Action creates a new handle, configured handles persist after save + reload → core-components/human-input-node-config.spec.ts

3.10 Data Operations (1.11.0)

• Data Operations component: unified JSON/Table/Text operations produce correct outputs per operation mode (Text→Message, Word Count→JSON override, JSON Select Keys, Table Filter) → core-components/data-operations-component.spec.ts
• Legacy operations components link/redirect to Data Operations (banner resolves the replacement, link filters the sidebar, old names still find the new component, legacy flows keep working) → core-components/data-operations-legacy-link.spec.ts

---

core-functionality/ — Core and Operational Logic

core-functionality/auth/ — Authentication and User Management

4.1 Login / Logout

• [-] Login with valid credentials
• [-] Login with invalid credentials — should display error message
• Logout — should redirect to login screen → core-functionality/auth/logout-flow.spec.ts
• [-] Auto-login enabled — should skip login screen
• [-] Auto-login disabled — should display login screen
• [-] Expired session — should redirect to login
• Session cleanup after logout — after logging out, navigating to / and reloading both stay on the login screen → core-functionality/auth/logout-flow.spec.ts

4.2 User Management (Admin)

• [-] Admin creates new user
• [-] Admin deactivates user
• [-] Admin activates inactive user
• [-] Admin renames user
• [-] Admin changes user password
• [-] Admin changes password — old password does not work after change
• [-] Isolation flow: user A cannot see user B's flows

4.3 Global Variables (API Keys)

• Create global variable
• [-] Use global variable in component (API key) → ui-ux/use-global-variable-in-component.spec.ts
• [!] Edit existing global variable — quarantined (#1235): clicking the variable's ag-grid row does not open the Update Variable modal, recurrent on the dailies of 2026-07-27 and 2026-08-03. Same surface and same shape as the provider-credential removal below (§7.5) — filed as one cause → ui-ux/global-variable-edit.spec.ts
• Delete global variable → ui-ux/global-variables-crud.spec.ts
• Create global variable of type "Generic" → ui-ux/global-variables-crud.spec.ts
• Credential variable value is hidden from the variable list → ui-ux/global-variables-crud.spec.ts
• Create global variable from Settings page → ui-ux/global-variable-edit.spec.ts

---

core-functionality/knowledge-ingestion-management/ — Upload, Processing and Vectors

5.1 File Upload

• Upload file via component → core-functionality/knowledge-ingestion-management/upload-via-component.spec.ts
• Upload files of different types (txt, pdf, json, py, wav) → core-functionality/knowledge-ingestion-management/file-types-upload.spec.ts
• File size limit → core-functionality/knowledge-ingestion-management/limit-file-size-upload.spec.ts
• File management page → core-functionality/knowledge-ingestion-management/files-page.spec.ts

5.2 Processing and Vectorization

• Split Text chunking of an ingested document → core-functionality/knowledge-ingestion-management/split-text-chunking.spec.ts
• Indexing in Vector Store — document available for query → core-functionality/knowledge-ingestion-management/vector-store-index-query.spec.ts
• Vector Store query returns relevant chunks for the prompt → core-functionality/knowledge-ingestion-management/vector-store-index-query.spec.ts
• Complete RAG pipeline (ingest → embed → store → retrieve → answer) → core-functionality/knowledge-ingestion-management/rag-pipeline.spec.ts

---

core-functionality/llm-agents/ — Agents and LLM Execution

⚠️ Tests in this section use SimpleAgentTemplatePage and are parameterized by model via models.json. Run npx playwright test tests/collect-models.spec.ts before executing these tests. See CLAUDE.md in this folder for the complete guide.

6.1 llm-agents/agent-component-regression.spec.ts — Agent Behavior Regression @stable

• Agent responds without connected tools
• Agent displays valid response and optionally reasoning steps
• Stop button interrupts agent execution
• Execution duration displayed after successful run
• Response displayed progressively in the Playground (streaming)
• Duration indicator displayed on canvas (node_duration_agent) after closing the playground
• Agent responds to multiple consecutive messages in the same session

6.2 Other execution tests

• [-] Agent displays reasoning steps in Playground → agent-reasoning-steps.spec.ts
• [-] Composio (tool integration for Agent) → composio.spec.ts
• Playground shows error when LLM run endpoint returns 500 (mocked invalid API key) → llm-agents/llm-invalid-api-key-ui.spec.ts
• Playground input remains usable after API error (mocked) → llm-agents/llm-invalid-api-key-ui.spec.ts
• [~] Agent stops when configured stop condition is reached → core-functionality/llm-agents/agent-max-iterations.spec.ts (max_iterations is the Agent's only configurable stop mechanism — no dedicated stop-condition field exists; see #824. Partial for the same reason as the bullet below: the stop itself is quarantined against #1264)
• [~] Agent stops when maximum number of iterations is reached → core-functionality/llm-agents/agent-max-iterations.spec.ts (partial: the stop assertion is quarantined (test.fixme, never @stable) against a live product failure — #1264. With max_iterations=1 on a task that needs several tool-calling iterations, the agent answers the task normally instead of returning Model call limits exceeded: run limit (1/1); the message renders, so this is a content failure, not a timeout. Reproduced on 1.12.0.dev18 locally, off CI load, on claude-haiku-4-5, claude-opus-5 and claude-opus-4-5, which rules out the mid-run backend wedge #1264's triage left open as a cover. The @stable causal control — a high limit finishes without the limit message — still runs, so what remains proven is that a high cap does not stop the agent, not that a low one does)
• Agent with multiple configured tools executes correctly → agent-multi-tool-selection.spec.ts
• Agent with configured timeout respects the limit (no product surface on 1.12.x — the Agent component exposes no timeout field: its inputs are max_iterations, max_tokens, n_messages, system_prompt, context_id, stream, and tools; Langflow's only configurable per-component timeouts live on the A2A Agent (timeout) and MCP tools (tool_execution_timeout), not the Agent, and the Language Model / chat models expose none either. The client/transport execution timeout that bounds any run is covered by ui-ux/execution-error-notification.spec.ts. Not automatable as written; #825, same class as #824)
• Connecting an external model in Agent drops the prior model selection (connection-mode isolation, prevents stale provider config) → llm-agents/agent-model-connection-isolation.spec.ts
• Flow with Agent saved and reopened → settings preserved → core-functionality/llm-agents/agent-config-persistence.spec.ts
• max_tokens truncates response as configured → llm-agents/agent-max-tokens.spec.ts (validated at token level via the Playground token-usage tooltip)

6.3 Memory and Context

• Memory Chatbot template loads with correct node and edge structure → llm-agents/memory-history-regression.spec.ts
• Message History retains context between messages in the same Playground session → llm-agents/memory-history-regression.spec.ts
• Session isolation: distinct session IDs have independent histories → llm-agents/memory-history-regression.spec.ts
• Without Message History, LLM does not retain context between messages → llm-agents/memory-history-regression.spec.ts
• n_messages parameter limits the number of retained messages → llm-agents/agent-n-messages-limit.spec.ts (bug reported fixed on 1.11.0.dev33 — parameter now respected; validated by deterministic message count)
• Agent uses custom context_id — continuity between session messages → agent-context-id-continuity.spec.ts
• Switching context_id isolates history between distinct sessions → agent-context-id-isolation.spec.ts

6.4 Tools and Integrations

• Agent with integrated external MCP tool executes action and returns result
• [-] Agent executes multiple tools in sequence → llm-agents/agent-multi-tool-selection.spec.ts (Test 3 — chained fetch→search, ordered tool_use assert; @stable gated on the clean baseline #818, per #827. #1378 bounded the run with a measured max_iterations cap after the unbounded Web Search payload blew the context up — still [-]: the cap raises the pass rate, it does not make the agent's convergence deterministic)
• Tool returns error — agent handles it and continues execution → core-functionality/llm-agents/agent-tool-error-handling.spec.ts
• Multiple connected tools — agent selects the correct one for each prompt → agent-multi-tool-selection.spec.ts
• Tool with invalid name — validation prevents execution with clear message → core-functionality/llm-agents/agent-tool-name-validation.spec.ts

6.5 Output and Reasoning

• [-] Inspect tools used by Agent in Playground → llm-agents/agent-tool-inspection.spec.ts (UI chip names the tool + persisted tool_use input/output; @stable gated on the clean baseline #818, per #827)
• Agent returns output in structured JSON format (output_schema) → agent-structured-output.spec.ts
• [-] Agent returns output in correctly rendered Markdown → llm-agents/agent-markdown-output.spec.ts
• Agent Instructions (system prompt) is respected in the model response → agent-system-prompt.spec.ts
• Input via direct field vs handle (ChatInput) — both work → core-functionality/llm-agents/agent-input-sources.spec.ts
• Empty response or model refusal — component does not crash → core-functionality/llm-agents/agent-empty-refusal-response.spec.ts
• Toggle add_current_date_tool works (enables/disables date tool) → agent-current-date-tool.spec.ts
• [~] handle_parsing_errors=False fails explicitly vs True auto-corrects → agent-parse-error-behavior.spec.ts (partially covered on 1.11: the field is present and togglable, but True/False are behaviorally identical — the field now only toggles ToolRetryMiddleware, and component-tool failures are converted to content by the hardcoded handle_tool_error=True before the middleware can observe them; the only live trigger (LLM-emitted malformed args) is non-deterministic, so the semantic difference is not deterministically testable — re-scope tracked in #496)
• Image passed via input handle is processed correctly → core-functionality/llm-agents/agent-multimodal-image-input.spec.ts
• Image attached in the Playground is processed by the Agent — the attachment renders in the user message and the reply describes it → core-functionality/llm-agents/general-bugs-agent-images-playground.spec.ts (@stable restored in #992 — the OpenAI quota that caused the #772 quarantine is back)

---

core-functionality/model-provider/ — Provider Management

⚠️ Provider configuration tests via Settings use SettingsPage. See helpers/provider-setup/ for the setup helpers of each provider.

7.1 Provider Collection and Validation

• Validate API keys of all providers via real call → collect-models.spec.ts
• Validate the running build can instantiate each provider's component (registry + build, not just the key) → collect-models.spec.ts
• Collect available models per provider via UI → collect-models.spec.ts
• Inactive providers appear as skipped in tests with reason → agent-component-regression.spec.ts
• Configure provider API key via Save Configuration (first setup) → collect-models.spec.ts
• Replace provider API key via Replace Configuration (existing key) → collect-models.spec.ts

7.2 OpenAI

• Configure OpenAI API key in Settings → Model Providers → core-functionality/model-provider/openai-provider.spec.ts
• Select GPT model in agent → core-functionality/model-provider/openai-provider.spec.ts
• Execute flow with OpenAI → core-functionality/model-provider/openai-provider.spec.ts
• Invalid API key error — display error message → core-functionality/llm-agents/provider-invalid-auth-error.spec.ts

7.3 Anthropic

• Configure Anthropic API key in Settings → Model Providers → core-functionality/model-provider/anthropic-provider.spec.ts
• Select Claude model in agent → core-functionality/model-provider/anthropic-provider.spec.ts
• Switch between Claude models (Sonnet, Haiku, Opus) → core-functionality/model-provider/anthropic-provider.spec.ts
• Invalid Anthropic API key error → core-functionality/llm-agents/provider-invalid-auth-error.spec.ts

7.4 Google Generative AI

• Configure Google API key in Settings → Model Providers → core-functionality/model-provider/google-provider.spec.ts
• Select Gemini model in agent → core-functionality/model-provider/google-provider.spec.ts
• Invalid Google API key error → core-functionality/llm-agents/provider-invalid-auth-error.spec.ts

7.5 Provider Management

• "Manage Model Providers" modal → llm-agents/modelProviderModal.spec.ts + llm-agents/model-provider-modal-actions.spec.ts
• Available provider count → llm-agents/model-provider-modal-actions.spec.ts
• Language Model component — configuration → llm-agents/language-model-regression.spec.ts
• Model Input component → llm-agents/modelInputComponent.spec.ts
• Add new provider via modal → llm-agents/model-provider-api-key.spec.ts (positive add validated via invalid-key rejection + Replace edit surface — a real re-add poisons a backend credential cache, see #505)
• Remove API key from existing provider — both paths validated. The UI path was quarantined (#1235) as a Global Variables permission-gate flake and is not one: the deletion already succeeded, and the spec then re-clicked the now-disabled header button through a phantom confirmation step. Fixed by asserting the header action is enabled and scoping the confirmation to a real dialog → llm-agents/remove-provider-api-key.spec.ts
• Per-model enable/disable toggle changes immediately and persists across reopen → llm-agents/model-provider-model-toggle.spec.ts
• Disabling a model in Settings removes it from a component model dropdown; re-enabling restores it → llm-agents/model-provider-model-toggle.spec.ts

7.6 Open-Source Providers

• Configure and execute flow with Ollama (local model) → model-provider/ollama-provider.spec.ts (both halves @stable again after #931: the lfx-ollama packaging gap that broke it on 07-23/24 is fixed upstream, a build-side bundle pre-flight now fails attributed instead of timing out blind for 30 s, the run-completion wait was rebuilt on the button-stop/button-send signal — the 07-15 failure mode — and the test model is derived from the instance instead of a hardcoded tag that could skip silently. Restored on 4 consecutive green manual.yml runs on 1.12.0.dev9, not on local evidence)
• [-] Configure and execute flow with Groq → model-provider/groq-provider.spec.ts (automated but not validatable on the tested image: the Groq component ships in the lfx-bundles distribution, which langflowai/langflow-nightly:latest does not install, so the spec's availability pre-flight skips it on every run — @stable removed, see #1039)
• [-] Configure and execute flow with Mistral → model-provider/mistral-provider.spec.ts (automated but not validatable on the tested image: same lfx-bundles packaging decision as Groq — @stable removed, see #1039)

7.7 Model Parameters (Agent)

• Maximum token count — response truncated as configured → llm-agents/agent-max-tokens.spec.ts
• [~] Maximum agent iterations → core-functionality/llm-agents/agent-max-iterations.spec.ts (partial — the stop assertion is quarantined against #1264; see §6.2 for the measurement)
• Use of custom context_id for memory isolation → agent-context-id-isolation.spec.ts
• Output formatting (JSON via output_schema, Markdown, plain text) → agent-structured-output.spec.ts

7.8 Unified Provider Setup — 1.11.0 additions

• OpenAI Compatible as a first-class unified model provider: setup with base URL + key, models discovered, usable by a flow → core-functionality/model-provider/openai-compatible-provider-setup.spec.ts (6 tests, all 6 @stable, validated on 1.12.0.dev19 against https://api.openai.com/v1 — the endpoint the issue itself suggests, so no new CI secret: the base URL defaults there and the bearer is the existing OPENAI_API_KEY, overridable via OPENAI_COMPATIBLE_TEST_BASE_URL / _API_KEY / _MODEL. Covered: the two-variable form with its asymmetric required/optional Save gate; the live-only catalog — this is the first provider with no static rows at all, so unconfigured it contributes 0 models, asserted differentially against Azure AI Foundry's seed catalog in the same run; an unresolvable .invalid base URL and a real endpoint with a bogus key both rejected on the validate-provider body (HTTP 200 + valid:false, DNS vs Authentication failed for the OpenAI-compatible endpoint) with nothing persisted; live discovery returning exactly the endpoint's own /v1/models id set, twice over (once per model type), polled on that terminal shape because the catalog is observable mid-registration; one discovered id — enabled for the provider by the test itself, since a default:false id never reaches the node's dropdown — running a Basic Prompting flow to a sentinel reply, gated on the persisted node carrying {name, provider: "OpenAI Compatible"} rather than a stored credential name, which upstream #14311 stopped writing (#1334); and configuring the provider through Settings persisting both variables — quarantined against LE-2124 (the concurrent POST /api/v1/variables/ writes dropped the key, silently) and un-quarantined on 1.12.0.dev19, where concurrent API writes answer 201/201 3/3 and the test passes 3/3 with its assertions unchanged)
• Azure AI Foundry in the unified provider setup: configuration accepts deployment names, provider appears configured → core-functionality/model-provider/azure-ai-foundry-provider-setup.spec.ts (6 tests, all validated against a live Azure resource on 1.12.0.dev15 — 3/3 clean --retries=0 --workers=1 runs, every test force-failed. 4 of them need no Azure account and carry the surface on every lane: the two-variable form, the Foundry-only deployment hint asserted differentially against OpenRouter, the read-only unconfigured panel, an unresolvable endpoint rejected with valid:false and nothing persisted, and a deployment name absent from every catalog accepted and stored as Azure AI Foundry::llm::<name>. The other two — real credentials configuring the provider through Settings, and a real inference addressed by the deployment name — now run there too: the three AZURE_AI_FOUNDRY_* secrets are wired into the daily shard step and manual.yml's Docker job by #1270, proven by a dispatch that reported 6 passed where the lane had said 4 passed, 2 skipped. They still skip with the concrete reason wherever the credentials are absent — a local run without the .env block, pr-validation.yml and the external-URL job, both unwired on purpose: #1216 keeps an unrelated PR off that account's health, and #1055 keeps us from storing our credentials in an instance we do not own)

---

core-functionality/observability-monitoring/ — Tracing, Logs and Metrics

8.1 Traces

• View execution traces → core-functionality/observability-monitoring/traces.spec.ts
• Trace API returns paginated transactions → core-functionality/observability-monitoring/traces-detail.spec.ts
• Trace displays latency of each component → core-functionality/observability-monitoring/traces-latency-tokens.spec.ts (API totalLatencyMs, Flow Activity latency column, per-span latency in the Trace Details modal)
• Trace displays tokens consumed → core-functionality/observability-monitoring/traces-latency-tokens.spec.ts (API totalTokens + Flow Activity token column)
• Single-trace API returns 404 for an unknown trace_id → traces-detail-single.spec.ts
• Single-trace API returns the full TraceRead contract with a non-empty span tree → traces-detail-single.spec.ts
• Single-trace API returns populated tokenUsage + modelName on the LLM span (OpenAI) → traces-detail-llm-span-populated.spec.ts
• Bulk delete traces API returns 404 for an unknown flow_id → traces-delete.spec.ts
• Bulk delete traces API clears all traces for the flow (204 + empty list) → traces-delete.spec.ts
• Bulk delete of a trace with a populated span tree cascades (204, no FK violation) — regression #13955 → traces-delete-cascade.spec.ts
• Trace list filter ?status=error returns only the failing trace; ?status=<unknown> returns 422 → traces-list-filters.spec.ts
• Trace list filter ?status=ok returns only the successful trace → traces-list-filters.spec.ts
• Trace list filter ?start_time pins the >= lower bound (past hits, future misses) → traces-list-filters.spec.ts
• Trace list filter ?query=<substring> filters by trace name, incl. 50-char sanitize cap → traces-list-filters.spec.ts
• Trace list filter ?session_id filters by the session passed at run time → traces-list-filters.spec.ts

8.2 Notifications

• System notifications — build-success entry shows in the notifications tab → notifications.spec.ts
• Execution error notification → ui-ux/execution-error-notification.spec.ts
• Outdated component notification → core-components/outdated-component-notification.spec.ts

8.3 User State

• Track user progress → core-functionality/project-management/user-progress-track.spec.ts
• User flow state cleanup → flow-functionality/user-flow-state-cleanup.spec.ts

8.4 Error Handling and Edge Cases

• Component that raises Python error → core-components/validate-raise-errors-components.spec.ts
• Flow with error displays appropriate message → core-functionality/observability-monitoring/flow-error-message.spec.ts
• Network error during execution → ui-ux/execution-error-notification.spec.ts
• Execution timeout — clear message to user → ui-ux/execution-error-notification.spec.ts (transport-timeout path: route.abort("timedout") → "Workflow run failed" / "Failed to fetch"; the distinct deployed-flow "Run timed out. Please try again." is out of scope — see #694)

---

core-functionality/playground/ — Chat, Rendering and Output Tests

9.1 Chat Interactions

• Open Playground → exercised by every @stable playground spec via playground-btn-flow-io
• Send text message → exercised by playground-ux.spec.ts, playground-message-edit.spec.ts, playground-session-nav.spec.ts and others
• Receive LLM response → exercised by all specs that send a message via ChatInput → ChatOutput echo flow
• Response streaming (SSE) → core-functionality/playground/playground-response-streaming-sse.spec.ts
• Response polling → api/flows/api-build-polling-response.spec.ts
• Direct response → api/flows/api-build-direct-response.spec.ts
• Playground UX (playground-ux) → playground/playground-ux.spec.ts
• Send empty message — send button stays enabled by design (only disabled while a file upload is in progress) → playground/playground-empty-message-send.spec.ts
• [-] Send message while response is in progress — should wait or queue → playground/playground-send-while-in-progress.spec.ts
• Attach image in chat — compact preview appears in input before sending → core-functionality/playground/playground-output-image.spec.ts
• Image rendered in user message bubble after sending → core-functionality/playground/playground-output-image.spec.ts
• Attach non-image file (.txt) in chat — preview tile renders (delete button visible, no <img>) → core-functionality/playground/playground-non-image-attachment.spec.ts
• Non-image file rendered in user message after sending — truncated filename appears, no image emitted → core-functionality/playground/playground-non-image-attachment.spec.ts
• Attach multiple images — one compact preview per file is shown in the input → core-functionality/playground/playground-attachments-management.spec.ts
• Remove one of two attachments — the remaining preview stays intact → core-functionality/playground/playground-attachments-management.spec.ts
• Send with multiple attachments — all images render in the user message → core-functionality/playground/playground-attachments-management.spec.ts
• Remove the only attachment — input returns to empty state and text-only send still works → core-functionality/playground/playground-attachments-management.spec.ts
• Swap attachment (remove A, attach B) — only B is sent → core-functionality/playground/playground-attachments-management.spec.ts
• ChatInput Input Text pre-fills the playground textarea on first open → core-functionality/playground/playground-input-text-prefill.spec.ts
• ChatInput Input Text re-pre-fills the textarea on a new session → core-functionality/playground/playground-input-text-prefill.spec.ts
• Pre-filled Input Text can be sent as the first message of the session → core-functionality/playground/playground-input-text-prefill.spec.ts
• [-] Attach and send an image on a live LLM flow (Basic Prompting) — the image renders in the chat messages → core-functionality/llm-agents/chatInputOutputUser-shard-0.spec.ts
• [-] Custom sender_name on Chat Input/Output is applied to a live LLM turn — messages render as chat-message-<custom name> after a default-label turn → core-functionality/llm-agents/chatInputOutputUser-shard-2.spec.ts

9.2 History and Session

• Configure custom session ID → core-functionality/playground/playground-session-id.spec.ts
• Switch session — messages are isolated per session → core-functionality/playground/playground-session-nav.spec.ts
• Edit user message — hover reveals edit button, saved changes replace original text → core-functionality/playground/playground-message-edit.spec.ts
• Cancel message edit — original text is preserved → core-functionality/playground/playground-message-edit.spec.ts
• Message edited in playground is reflected in Session Logs → core-functionality/playground/playground-message-edit.spec.ts
• Clear chat removes all messages from Default Session (clear-chat-option via header menu) → core-functionality/playground/playground-session-clear.spec.ts
• Clear full session history (Default session) → playground/playground-clear-history.spec.ts
• Delete user-created session → playground/playground-clear-history.spec.ts
• History persists when reopening Playground → llm-agents/memory-history-regression.spec.ts, core-functionality/playground/playground-history-persist.spec.ts
• Rename unavailable for the Default Session → core-functionality/playground/playground-session-rename.spec.ts
• Rename unavailable for a session with no messages → core-functionality/playground/playground-session-rename.spec.ts
• Rename available and functional for a session with messages (Enter confirms, Escape cancels) → core-functionality/playground/playground-session-rename.spec.ts
• Create new session via new-chat button → core-functionality/playground/playground-session-nav.spec.ts
• Switch between sessions via session selector sidebar → core-functionality/playground/playground-session-nav.spec.ts
• Open Message Logs via session more-menu → core-functionality/playground/playground-message-logs.spec.ts
• Delete messages inside Message Logs table → core-functionality/playground/playground-message-logs.spec.ts
• Select individual session checkbox → reveals bulk-delete-button → core-functionality/playground/playground-bulk-delete.spec.ts
• Select all non-default sessions via select-all-checkbox → core-functionality/playground/playground-bulk-delete.spec.ts
• Bulk delete selected sessions → Default Session preserved → core-functionality/playground/playground-bulk-delete.spec.ts

9.3 Advanced Playground Features

• Playground fullscreen mode → playground/playground-fullscreen.spec.ts
• Shareable Playground — URL generation validated (switch enables sharing, href matches /playground/uuid) → playground/playground-shareable-url.spec.ts
• [!] Voice mode (voice assistant) → ui-ux/voice-assistant.spec.ts (all tests unconditionally skipped — spec is a stub)
• Stop button in Playground → core-functionality/playground/stop-button-playground.spec.ts

9.4 Output Modal

• Copy component output → core-functionality/playground/output-modal-copy-button.spec.ts
• Copy button in output → core-functionality/playground/output-modal-copy-button.spec.ts

9.5 Structured Data Output

• JSON Data output renders as code block → core-functionality/playground/playground-output-data.spec.ts
• DataFrame output renders as Markdown table → core-functionality/playground/playground-output-data.spec.ts

9.6 Human-in-the-Loop (1.11.0)

• Human Input suspends the run server-side, decision card renders in the Playground, Approve routes only the approved branch and the run leaves the suspended state; Reject routes only the reject branch → core-functionality/playground/human-input-pause-resume.spec.ts
• A suspended Human Input run is recoverable after a page reload (durable execution outliving the tab)

---

core-functionality/project-management/ — Project and Folder Management

10.1 Folder CRUD

• Create new folder → core-functionality/project-management/folder-crud.spec.ts
• Rename folder → core-functionality/project-management/folder-crud.spec.ts
• Delete empty folder → core-functionality/project-management/folder-crud.spec.ts
• Delete folder with flows inside → core-functionality/project-management/folder-crud.spec.ts
• Integrity after deletion — the deleted folder leaves the sidebar immediately, the page stays functional, and a sibling folder is untouched and still clickable → core-functionality/project-management/folder-deletion-integrity.spec.ts
• Create folder after deleting all folders — creating a folder right after a deletion works (no stale-cache collision) → core-functionality/project-management/folder-deletion-integrity.spec.ts
• [-] Deleting every folder lands on the empty-project screen (sidebar empty message + new_project_btn_empty_page) → core-functionality/project-management/folder-deletion-integrity.spec.ts (@destructive — account-wide wiper, runs only in the low-concurrency lane via PW_DESTRUCTIVE=1, see #1010; stays [-] permanently, since [x] requires @stable and @destructive must never carry it — the pair would mean "runs nowhere")
• [-] Upload flow by drag-and-drop to folder — dropping a collection file imports one flow per entry; dropping a single flow file imports exactly one → flow-functionality/dragAndDrop.spec.ts
• [-] Move flow to another folder

10.2 Folder Navigation

• [-] Navigate between folders → core-functionality/project-management/flow-navigation-between-folders.spec.ts
• [-] Search flow by name filters results correctly
• [-] Folders in navigation sidebar

---

core-functionality/templates/ — Predefined Flow and Component Models

11.1 Basic Templates

• [-] Basic Prompting (OpenAI)
• [-] Basic Prompting (Anthropic)
• [-] Simple Agent (OpenAI)
• [-] Simple Agent (Anthropic)
• [-] Simple Agent with memory
• [-] Vector Store RAG
• Memory Chatbot
• [-] Basic Prompting (OpenAI) → core/integrations/Basic Prompting.spec.ts
• [-] Basic Prompting (Anthropic) → core/integrations/Basic Prompting Anthropic.spec.ts
• [-] Simple Agent (OpenAI) → core/integrations/Simple Agent.spec.ts
• [-] Simple Agent (Anthropic) → core/integrations/Simple Agent Anthropic.spec.ts
• [-] Simple Agent with memory → core/integrations/Simple Agent Memory.spec.ts
• [-] Vector Store RAG → core/integrations/Vector Store.spec.ts
• Memory Chatbot → llm-agents/memory-history-regression.spec.ts

11.2 Content Generation Templates

• [-] Blog Writer
• [-] Instagram Copywriter
• [-] Twitter Thread Generator
• [-] SEO Keyword Generator
• [-] Portfolio Website Code Generator
• [-] SaaS Pricing

11.3 Analysis and Processing Templates

• [-] Document QA
• [-] Invoice Summarizer
• [-] Financial Report Parser
• [-] Image Sentiment Analysis
• [-] Text Sentiment Analysis
• [-] Youtube Analysis

11.4 Agent Templates

• [-] Dynamic Agent
• [-] Hierarchical Agent
• [-] Sequential Task Agent
• [-] Social Media Agent
• [-] Travel Planning Agent
• [-] Market Research
• [-] Research Translation Loop
• [-] Pokedex Agent
• [-] Price Deal Finder
• [-] News Aggregator

11.5 Advanced Templates

• [-] Custom Component Generator
• [-] Prompt Chaining
• [-] Decision Flow
• [-] Similarity
• [-] MCP Server (starter projects)

---

core-functionality/a2a/ — Agent-to-Agent Protocol (1.11.0)

⚠️ Every bullet below needs LANGFLOW_A2A_ENABLED=true on the instance — the flag is off by default (lfx/services/settings/groups/mcp.py), and with it off all three /api/v1/a2a/* routes answer 404 and the flow editor's Agent tab only renders "A2A is turned off on this server". Surface map, testability decisions and the full out-of-scope list: docs/core-functionality/a2a/a2a-coverage-scope.md (scoping issue #1195, upstream langflow-ai/langflow#13831, Jira epic LE-1588). Numbering starts at 16 because §12–§15 are already taken; the section sits here, with its area, on purpose.

16.1 A2A Server

• Agent card served for a published flow — GET /api/v1/a2a/{flow_id}/.well-known/agent-card.json returns protocolVersion="0.3.0", url ending in /api/v1/a2a/{flow_id}/jsonrpc, capabilities.streaming=true, defaultInputModes=["application/json"], skills[0].id === flow_id, skills[0].tags=["langflow"] and an inputSchema object → core-functionality/a2a/a2a-server-agent-card.spec.ts
• Card overrides applied — a2a_card_overrides (name / version / description / tags / examples) change exactly those card fields and nothing else → core-functionality/a2a/a2a-server-agent-card.spec.ts
• Card gated on publication state — a2a_enabled=false, flow_type=workflow and an unknown flow id each return 404 (indistinguishable from an unmounted route, by design) → core-functionality/a2a/a2a-server-agent-card.spec.ts
• Agent discovery list — GET /api/v1/a2a/agents lists the published flow with a cardUrl that resolves 200, omits a flow_type=workflow flow and an agent flow with a2a_enabled=false, and drops the row when the flow is unpublished (owner-scoped, not a cross-user directory) → core-functionality/a2a/a2a-server-discovery.spec.ts
• JSON-RPC message/send round-trip — a per-run sentinel sent to a Chat Input→Chat Output passthrough comes back in the task's artifact text with state completed (no LLM involved) → core-functionality/a2a/a2a-server-jsonrpc-message-send.spec.ts
• JSON-RPC error envelopes — an unknown method returns -32601 and a malformed envelope -32600/-32700, both over HTTP 200 (JSON-RPC-level errors are not HTTP errors here) → core-functionality/a2a/a2a-server-jsonrpc-message-send.spec.ts
• Multi-turn context continuity — the first message/send response carries a server-minted contextId; reusing it on a second call returns the same contextId with a new task id and lands in the same stored session (session_id is the composite <uuid>:<contextId> in GET /api/v1/monitor/messages, carrying both turns as User/Machine pairs), while a call without it mints a different one → core-functionality/a2a/a2a-server-multi-turn-context.spec.ts
• Task lifecycle — tasks/get reads the message/send task back with the same artifactId and status.timestamp (a read-back, not a re-run); an unknown id is -32001 "Task not found" and a cancel on a finished task -32002 "Task cannot be canceled" with the stored state untouched, both over HTTP 200; a task id cancelled through another flow's endpoint is -32001 (never -32002, which would confirm it exists); and a message/stream run cancelled mid-flight reports canceled from both tasks/cancel and tasks/get → core-functionality/a2a/a2a-server-tasks-lifecycle.spec.ts
• API-key auth gate on the JSON-RPC endpoint — with the flow's project set to auth_type=apikey, the card advertises the x-api-key scheme in securitySchemes, a call with no header returns 401 "API key required", a wrong key 401 "Invalid API key", and the owner's key 200 + completed (the gate LE-2081 lives behind; auth derives from the project, not the flow). The same flow is asserted before and after it is moved into the restricted project — one id crossing the boundary, so the 401 cannot be blamed on the flow, the graph, the server flag or the environment → core-functionality/a2a/a2a-server-auth-apikey.spec.ts
• Agent tab publish flow — a blank flow shows Unavailable with the copy "Add a chat input and output to serve this flow." and cannot publish; adding Chat Input + Chat Output (unwired — the two node types are the gate) enables agent-publish-switch; publishing shows status Live and the agent-card-url that 404d as a draft now fetches 200; editing Name and adding a tag then pressing agent-save updates agent-card-name and changes name / skills[0].name / skills[0].tags on the card the API serves. The "Agent updated" toast is not asserted — measured transient (<3 s), so the durable pair is the status chip plus the served card → core-functionality/a2a/a2a-server-agent-tab-publish.spec.ts
• [~] Agent tab "Try it" panel — a sentinel sent from the panel over the live endpoint appears in agent-transcript twice (user turn + the agent's echo), the state reaches completed, the turn counter reads 1 turn and a Reset control is present → core-functionality/a2a/a2a-server-agent-tab-try-it.spec.ts. Partial: "View JSON-RPC exchange" is not covered because it is not implemented — agentTab.viewExchange appears exactly once per locale bundle in the shipped frontend (the dictionary entry) with zero call sites, and the string is absent from the DOM after a completed turn (measured on 1.12.0.dev14, #1244); pending an upstream question
• Non-owner cannot publish a flow — PATCH /api/v1/flows/{id} flipping a2a_enabled returns 403 "Cannot change a2a_enabled of a flow you do not own." (not automatable: needs two real users, and per-test user isolation is impossible under AUTO_LOGIN — measured in #1010)
• Disabled-server state — all three /api/v1/a2a/* routes 404 and the Agent tab renders "A2A is turned off on this server. Set LANGFLOW_A2A_ENABLED=true…" (out of scope by lane decision on #1195: the flag is on in every lane, and an off-lane cannot coexist with it in the same run; revisit only as a dedicated PW_A2A_OFF=1 lane)
• Push notification config and delivery — tasks/pushNotificationConfig/{set,get,list,delete} (out of reach: proving delivery needs a receiver with an inspectable inbox, which the self-hosted go-httpbin echo endpoint does not provide — LE-1706)
• JWS-signed agent cards (LE-1718) and rate limiting on the public endpoints (LE-1701) (out of reach: no URL-observable surface for the signature; driving the global v1 rate limits would make every parallel lane flaky)

16.2 A2A Client

• A2A Agent component, Internal mode — the mode=Internal dropdown lists the locally published agent and a run returns a Response containing the sentinel the published passthrough flow echoes (no LLM on either side). The node opens in External, so the mode tab must be clicked for the picker to exist at all, and the dropdown is folder-scoped (list_a2a_agents_by_flow_folder), so both flows live in the same project. Asserted through a wired Chat Output in the playground, not the node's output modal: output-inspection-response-a2aagent reads "No Data Available" even after a successful run whose stream carried outputs.response.message (measured 3× on 1.12.0.dev18, candidate product defect recorded in the spec doc) → core-functionality/a2a/a2a-client-agent-internal.spec.ts
• A2A Agent component, External mode — pointed at this instance's own card URL, the agent_card display renders the card chips (name; "Requires an API key" when the project is restricted) and a run returns the echoed sentinel; @regression for LE-1845 (NameError: name 'call_a2a_agent' is not defined). Blocked until a loopback self-call is allowed past Langflow's SSRF layer (LE-1904 class) — if it cannot be, LE-1845 stays uncovered
• A2A Agent used as a Tool — an Agent with the A2AAgent wired as a tool (tool_mode) calls the published agent and the reply reaches the playground; @regression for LE-1963 (self.user_id is None → "badly formed hexadecimal UUID string" on tool-approval resume). The only LLM-dependent bullet of the area — --workers=1 + models.json

---

flow-functionality/ — Graph Execution, Drag-and-Drop and JSON

12.1 Create Flow

• Create blank flow → flow-functionality/create-blank-flow.spec.ts
• Create flow from template → flow-functionality/create-flow-from-template.spec.ts
• Create flow by duplicating an existing one → flow-functionality/duplicate-flow.spec.ts
• Create flow via JSON file import → flow-functionality/export-import-flow.spec.ts

12.2 View and Edit Flow

• Rename flow via editor header → flow-functionality/flow-rename-header.spec.ts
• Rename flow and verify on main page listing → core-functionality/project-management/edit-flow-name.spec.ts
• Edit flow name and description → core-components/edit-name-description-node.spec.ts
• Flow auto-save on changes → flow-functionality/auto-save-off.spec.ts
• Flow settings → core-functionality/project-management/flowSettings.spec.ts

12.3 Delete Flow

• Delete individual flow → ui-ux/actionsMainPage-shard-1.spec.ts
• Delete multiple flows (bulk actions) → core-functionality/project-management/bulk-actions.spec.ts
• Shift-click range select + Ctrl/Cmd-click multi-select on main page → core-functionality/project-management/bulk-actions.spec.ts
• Bulk download selected flows → core-functionality/project-management/bulk-actions.spec.ts
• Confirm deleted flow does not appear in listing (after bulk delete) → core-functionality/project-management/bulk-actions.spec.ts

12.4 Export / Import Flow

• Export flow as JSON → flow-functionality/export-import-flow.spec.ts
• Exported JSON contains valid data.nodes structure → flow-functionality/export-import-flow.spec.ts
• Import flow via JSON file upload (drag-drop + upload button) → flow-functionality/export-import-flow.spec.ts
• [-] Import flow with outdated components → flow-functionality/import-outdated-flow.spec.ts
• Import invalid JSON — should display error message → flow-functionality/import-invalid-json.spec.ts

12.5 Flow Operations

• Lock flow — prevents editing → flow-functionality/lock-flow.spec.ts
• Unlock flow → flow-functionality/flow-lock.spec.ts
• [!] Move flow between folders via API — quarantined (#932): under concurrent writes PATCH /api/v1/flows/{id} answers 500 (sqlite3.OperationalError: database is locked on UPDATE flow SET folder_id) and the flow does not move; 14/24 at 2 concurrent clients, 0/30 serial. Same root cause as #965, not a separate one — the daily artifact shows the failing assert is expect(patchRes.status()).toBe(200) receiving 500, not a stale folder_id. Product defect tracked under LE-2020; the 200 assertion is unchanged → api/flows/api-folders-crud.spec.ts
• Publish flow → flow-functionality/publish-flow.spec.ts
• Save flow components as template → core-components/saveComponents.spec.ts

12.6 Flow Execution

• [!] Run Flow component executes another flow — runs again after the #966 quarantine lift, but not @stable while the upstream New Flow dead-click defect (LE-2019) is open; the shared helper gates on the flows list having rendered so the suite stays out of the broken window → flow-functionality/run-flow.spec.ts
• Run a flow from the canvas — terminal-node run builds the whole graph; all nodes reach build success and output is produced → flow-functionality/flow-execution-canvas.spec.ts
• Stop building flow → flow-functionality/stop-building.spec.ts
• [!] Playground button disabled with empty flow — needs review → regression/flow-functionality/generalBugs-shard-3.spec.ts (test skipped: assertion was a no-op, current Langflow behavior to confirm)

---

mcp/ — Model Context Protocol

⚠️ Tests that execute agents via MCP must use SimpleAgentTemplatePage and models.json. See CLAUDE.md in this folder for the complete guide.

mcp/client/ — Tool and Context Consumption

13.1 MCP Client

• Configure connection with external MCP server (stdio or HTTP) → mcp/client/mcp-client-regression.spec.ts
• List available tools via MCP protocol → mcp/client/mcp-client-regression.spec.ts
• Execute MCP server tool and receive result in flow → mcp/client/mcp-client-regression.spec.ts
• MCP server connection error — unreachable server produces empty tool dropdown → mcp/client/mcp-client-regression.spec.ts
• Configure connection via HTTP form tab → mcp/client/mcp-client-regression.spec.ts
• [-] configureMcpServer helper registers an MCP server via the HTTP form → core-components/configure-mcp-and-custom-component.spec.ts
• Execute numeric tool with inputs and verify result → mcp/client/mcp-client-regression.spec.ts
• Duplicate MCP server registration returns 409 Conflict → mcp/client/mcp-server-registration-status-codes.spec.ts
• Deleting a non-existent MCP server returns 404 Not Found → mcp/client/mcp-server-registration-status-codes.spec.ts
• [-] Agent uses MCPTools as tool and calls echo via MCP → mcp/client/mcp-client-agent.spec.ts (automated, not @stable — no lane runs it, so it cannot report a regression; #1371)
• Gemini × MCP tool-calling regression — agent invokes the echo MCP tool (regression for fixed upstream #440) → mcp/client/mcp-client-agent-gemini-tool-regression.spec.ts
• List available resources via MCP protocol (client not-implementable on 1.11.x — MCPTools component and v2 client API expose tools only; server-side resources covered in §14.1 → mcp/server/mcp-server-resources.spec.ts)
• Consume resource URI and inject content into flow (client not-implementable on 1.11.x — no client resource surface; server-side read covered in §14.1 → mcp/server/mcp-server-resources.spec.ts)

---

mcp/server/ — Resource and Tool Provider

14.1 MCP Server

• MCP Server tab in flow → mcp/server/mcp-server-tab.spec.ts
• Add MCP server via modal → mcp/server/mcp-server-tab.spec.ts
• Starter project with MCP → mcp/server/mcp-server-starter-projects.spec.ts
• Flow exposed as MCP server — verify generated endpoint → mcp/server/mcp-server-protocol.spec.ts
• Execute MCP server tool via MCP protocol → mcp/server/mcp-server-protocol.spec.ts
• Register an external MCP server through the stdio form — command + args resolves the server's real tools into the MCPTools node → mcp/server/mcp-server.spec.ts
• Add-server modal fields persist across save → reopen-for-edit — stdio (name, command, 4 args, 2 env pairs) and HTTP/SSE (name, URL, 2 headers, 2 env pairs) → mcp/server/mcp-server.spec.ts
• Tool list refreshes when a registered server is edited to run a different package → mcp/server/mcp-server.spec.ts
• stdio command must be a single executable — a command with an embedded argument is refused and the same registration split into command + args is accepted (upstream hardening #14073; #1091) → mcp/server/mcp-server.spec.ts
• The project's own Streamable HTTP endpoint registers as an MCP server and exposes its flows as tools → mcp/server/mcp-server.spec.ts
• [-] Resource exposed by server is accessible via URI — flow files are exposed as MCP resources: resources/list is @stable; resources/read blocked by a live Langflow regression on 1.12.x (AttributeError: 'str' object has no attribute 'hex', filed upstream LE-2012) — kept as a guard, not promoted → mcp/server/mcp-server-resources.spec.ts
• Prompt exposed by server returns correct template (no product surface on 1.11.x — MCP server prompts/list returns []; #829)

---

ui-ux/ — Visual Interface, Canvas and Design System

15.1 Component Sidebar

• Search component by name — matching component listed AND non-matching one hidden, case-insensitive → ui-ux/sidebar-search-and-filter.spec.ts
• [~] Hover over component shows tooltip/preview — no product surface on 1.12.0.dev6: hovering a sidebar card renders zero [role="tooltip"] elements, zero Radix poppers, no title and no aria-describedby; the only hover affordance is the + button, already covered by core-components/componentHoverAdd.spec.ts (#937)
• Keyboard search (keyboard shortcut) — / focuses the search, Tab reaches a result, Space/Enter add that exact component → ui-ux/keyboardComponentSearch.spec.ts
• Filter components by category — disclosure collapse/expand and non-matching categories removed while filtering → ui-ux/sidebar-search-and-filter.spec.ts
• [~] Sidebar shows correct provider count — no count surface on 1.12.0.dev6: the sidebar renders no numeric badge or count text anywhere (the count lives on Settings → Model Providers, §7). What exists is grouping under disclosure-bundles-<provider>, covered by ui-ux/sidebar-search-and-filter.spec.ts (#937)

15.2 Add Components to Canvas

• Drag component from sidebar to canvas — node lands at the drop position → ui-ux/sidebar-add-component.spec.ts
• Double-click in sidebar adds component to canvas → ui-ux/sidebar-add-component.spec.ts
• Hover + click "+" button adds component to canvas → core-components/componentHoverAdd.spec.ts
• Added component appears with default settings — every field value compared against the GET /api/v1/all catalog template → ui-ux/sidebar-add-component.spec.ts

15.3 Component Connections

• Connect two compatible components — clicking the Chat Input source handle then the Chat Output target handle creates exactly one edge, persisted to data.edges; repeating the pair does not duplicate it → flow-functionality/canvas-connect-components.spec.ts
• Prevent connection between incompatible types — a DataFrame output (Split Text chunks) into a Message input (Chat Output) creates no edge, while the Message→Message pair in the same test does (positive control); target-to-target is separately covered as invalid topology → flow-functionality/canvas-connect-components.spec.ts
• Delete edge/connection — right-clicking the edge context menu and choosing the destructive item removes the edge from the canvas and from data.edges → flow-functionality/canvas-edge-reconnect.spec.ts
• Filter edges by data type — clicking an input handle filters the sidebar to compatible sources (url → string sources, headers → Data sources), and the legacy/beta toggles expand that set → ui-ux/filterSidebar.spec.ts
• Reconnect existing edge — after deleting it, clicking the same two handles restores exactly one edge in the canvas and in the flow → flow-functionality/canvas-edge-reconnect.spec.ts

15.4 Node Manipulation

• Delete component from canvas via Backspace key → core-components/componentDelete.spec.ts
• Delete component from canvas via node options (...) menu → core-components/componentDelete.spec.ts
• Copy and paste ChatOutput component (Ctrl+C / Ctrl+V) → flow-functionality/canvas-copy-paste.spec.ts
• Copy and paste Prompt Template (component with dynamic ports) (Ctrl+C / Ctrl+V) → flow-functionality/canvas-copy-paste.spec.ts
• Canvas keyboard shortcuts — Duplicate/Delete/Copy/Paste/Cut/Undo/Redo each act on the selected node, with the selection re-gated before every keypress → ui-ux/langflowShortcuts.spec.ts
• Minimize component on canvas — the options menu collapses the node (every handle gains no-show, height shrinks) and persists data.showNode = false; the item swaps to Expand, which restores both; four further minimize/expand cycles keep both states correct and the persisted showNode in step (#1290) → ui-ux/minimize.spec.ts
• Move component within canvas — dragging a node by its title moves it on canvas and the new coordinates reach the backend (GET /api/v1/flows/{id} position matches the rendered transform) → flow-functionality/canvas-move-node.spec.ts
• Select multiple components via box selection — a Shift+drag marquee enclosing two separated nodes takes .react-flow__node.selected from 0 to 2, while a marquee drawn away from them selects nothing (negative control) → flow-functionality/canvas-multiselect.spec.ts
• Delete multiple selected components (marquee box selection) → core-components/componentDelete.spec.ts, flow-functionality/canvas-multiselect.spec.ts
• Deselect node by clicking on empty canvas area — clicking .react-flow__pane clears a selection asserted present first → flow-functionality/canvas-deselect-node.spec.ts
• Deselect node via Escape → flow-functionality/canvas-deselect-node.spec.ts

15.5 Canvas Zoom and Navigation

• Zoom in / Zoom out → ui-ux/canvas-zoom-navigation.spec.ts
• Fit View centers nodes → ui-ux/canvas-zoom-navigation.spec.ts
• Fit View button in toolbar → ui-ux/canvas-zoom-navigation.spec.ts
• Scroll to navigate canvas — on 1.12 the wheel zooms anchored at the pointer (no scroll-to-pan surface) → ui-ux/canvas-zoom-navigation.spec.ts
• [~] Minimap — feature flag-gated

15.6 Grouping

• Create component group → core-components/nested-grouping-regression.spec.ts
• Ungroup components → core-components/nested-grouping-regression.spec.ts
• Expand/collapse group → core-components/nested-grouping-regression.spec.ts

15.7 Freeze and State

• Freeze component — a frozen component serves its cached output instead of recomputing → flow-functionality/freeze-and-state.spec.ts
• Freeze path — freezing a component also freezes every component upstream of it → flow-functionality/freeze-and-state.spec.ts
• Unfreeze component — releases the whole path and the component recomputes → flow-functionality/freeze-and-state.spec.ts

15.8 Sticky Notes

• Add sticky note — the canvas-add-note-button canvas control places a note at the default 280×140 and the flow gains a node with type: "noteNode" → ui-ux/sticky-notes.spec.ts
• Edit sticky note text → ui-ux/edit-sticky-note-text.spec.ts
• Change sticky note color — the picker offers all seven presets; choosing rose repaints the note (--note-rose inline style) and persists template.backgroundColor: "rose" → ui-ux/sticky-notes.spec.ts
• Resize sticky note — dragging the bottom-right resize handle grows the note past 280×140 and the new dimensions persist to the node's width/height → ui-ux/sticky-notes.spec.ts
• Delete sticky note — removed via the options menu and via Backspace, gone from the canvas and from the persisted flow; deleting one of two leaves exactly one → flow-functionality/canvas-sticky-note-delete.spec.ts

15.9 Right-Click and Menus

• [~] Context menu via right-click on canvas — no product surface on 1.12.0.dev8: a right-click on .react-flow__pane dispatches a contextmenu event that reaches document with defaultPrevented === false (so the browser's native menu is what opens) and renders zero [role="menu"] / [role="listbox"] / Radix popper elements, with or without a node selected — Langflow does not wire ReactFlow's onPaneContextMenu. That absence is now guarded negatively (the pane right-click dismisses an open node menu, opens nothing, and leaves the selection intact) → ui-ux/right-click-dropdown.spec.ts. Edge menus (edge-context-menu-trigger) are a left-click affordance and belong to §15.3 (#945, #1027)
• Context menu via right-click on component — one right-click selects the node AND opens its options menu with the exact ordered item contract (Save/Duplicate/Copy/Docs/Minimize/Freeze/Download/Delete), Escape closes it, and choosing Duplicate from that menu adds the node → ui-ux/right-click-dropdown.spec.ts
• Main menu actions — the header account menu lists all nine menu_*_button items, its version row matches GET /api/v1/version, the four external items carry their documented href/target=_blank, Escape closes it, and the Settings item routes to /settings → ui-ux/main-menu-actions.spec.ts

15.10 Settings and UI Configuration

• Access Settings page — profile menu opens Settings, the sidebar lists General/Model Providers/Shortcuts/Messages, and each section renders its own content (General shows the Language + Profile Picture groups) → ui-ux/settings-navigation.spec.ts, ui-ux/settings-general-section.spec.ts
• Message history settings — Settings → Messages grid keeps the 11-column contract, renders messages oldest-first (1.12 get_messages defaults to order=ASC) and the sender "Equals User" filter narrows/restores the row set → ui-ux/settings-message-history.spec.ts
• Change appearance/theme settings — dark/light toggle updates #body.dark class → ui-ux/settings-theme-toggle.spec.ts
• Keyboard shortcuts work in editor — Duplicate/Delete/Copy/Paste/Cut/Undo/Redo each act on the selected node (node count asserted after every keypress) → ui-ux/langflowShortcuts.spec.ts
• [~] All documented shortcuts work — all 27 defaultShortcuts rows are listed with a non-empty key binding in Settings → Shortcuts (ui-ux/settings-navigation.spec.ts), and 7 of them are exercised on canvas (ui-ux/langflowShortcuts.spec.ts) plus 1 rebound end-to-end (ui-ux/settings-shortcuts-edit.spec.ts); the remaining 20 (API, Docs, Download, Play, Group, Minimize, Freeze, Save, Code, Update, Controls, sidebar search, …) are not exercised yet
• Edit a keyboard shortcut (Duplicate → Ctrl/Cmd+Alt+U) persists to the table and the new combination triggers the action on canvas → ui-ux/settings-shortcuts-edit.spec.ts
• API Keys table renders created_at/expires_at in the viewer's local timezone (UTC→local), shows "Never" for unused keys and ∞ for no-expiry keys (PR #13471) → ui-ux/api-keys-timezone-display.spec.ts

---

Coverage Summary — Test Automation Coverage

Validated = test carries the @stable tag. Needs validation = automated but not yet @stable (bug, flake under investigation, or pending team review).

Module	Total	Validated [x]	Needs validation [-]	Partial [~]/[!]	Not automated [ ]
api/flows/ — REST API	28	27	0	1	0
core-components/ — Component Config	27	24	3	0	0
core-components/ — Core Components	91	87	3	0	1
core-functionality/auth/	21	7	13	1	0
core-functionality/knowledge-ingestion/	8	8	0	0	0
core-functionality/llm-agents/	40	30	5	3	2
core-functionality/model-provider/	34	31	2	1	0
core-functionality/observability-monitoring/	24	24	0	0	0
core-functionality/playground/	52	47	3	1	1
core-functionality/project-management/	12	6	6	0	0
core-functionality/templates/	41	2	39	0	0
core-functionality/a2a/	18	11	0	1	6
flow-functionality/	28	24	1	3	0
mcp/client/	13	9	2	0	2
mcp/server/	12	10	1	0	1
ui-ux/ — Canvas	44	40	0	4	0
ui-ux/ — Settings	7	6	0	1	0
TOTAL	500	393 (79%)	78 (16%)	16 (3%)	13 (3%)

Note: Validated [x] counts checklist bullets, not test() calls. The @stable tag is per-test(), and a single @stable test may map to several bullets via test.step() (e.g. the agent suite covers 7 bullets). The canonical list of @stable test() calls is in Phase 0 — Validated below.

---

Implementation Roadmap

---

🟢 Phase 0 — Validated

467 test() calls carrying the @stable tag, distributed across 185 spec files. Run weekly by the stable workflow. New specs are merged with all tests tagged @stable; the tag is removed per-test during weekly triage when a failure is classified as a test bug — so a spec may end up with a mix of tagged and untagged tests over time.

api/flows/

• direct event_delivery streams build events inline (no job_id) and echoes the input → api-build-direct-response.spec.ts
• direct is distinct from the job_id path: streaming delivery returns a job_id → api-build-direct-response.spec.ts
• polling is the two-step path: POST returns a job_id shell (no inline events) → api-build-polling-response.spec.ts
• the poll loop drains the build to completion across repeated GET /events calls → api-build-polling-response.spec.ts
• API Request component — include_httpx_metadata=true adds request headers to output → api-component-regression.spec.ts
• API Request component — timeout error returns status_code 500 with error field → api-component-regression.spec.ts
• POST /api/v1/custom_component returns valid component structure → api-custom-component-creation.spec.ts
• POST /api/v1/custom_component with invalid code returns error → api-custom-component-creation.spec.ts
• GET /api/v1/all includes component types → api-custom-component-creation.spec.ts
• POST /api/v1/custom_component without auth returns 401 or 403 → api-custom-component-creation.spec.ts
• POST creates flow and returns ID → api-flows-crud.spec.ts
• GET lists flows and includes the created one → api-flows-crud.spec.ts
• GET by ID returns correct flow → api-flows-crud.spec.ts
• PATCH updates flow name and description → api-flows-crud.spec.ts
• DELETE removes flow and returns 200 → api-flows-crud.spec.ts
• GET after DELETE returns 404 → api-flows-crud.spec.ts
• GET non-existent flow returns 404 → api-flows-crud.spec.ts
• POST with missing name returns 422 → api-flows-crud.spec.ts
• deleted flow does not appear in flows listing → api-flows-crud.spec.ts
• POST creates folder and returns ID and name → api-folders-crud.spec.ts
• GET lists folders and includes the created one → api-folders-crud.spec.ts
• GET /health_check returns 200 with status ok → api-health-check.spec.ts
• GET /health_check returns db ok → api-health-check.spec.ts
• GET /health_check responds within 5 seconds → api-health-check.spec.ts
• GET /health_check response has correct content-type → api-health-check.spec.ts
• POST /api/v1/flows/ with invalid Bearer token returns 401, 403, or 422 → api-invalid-key.spec.ts
• GET /api/v1/flows/ without Authorization header returns 401 or 403 → api-invalid-key.spec.ts
• GET /api/v1/flows/{id} with invalid Bearer token returns 401 or 403 → api-invalid-key.spec.ts
• POST /api/v1/run/{id} with invalid x-api-key returns 401 or 403 → api-invalid-key.spec.ts
• DELETE /api/v1/flows/{id} without Authorization header returns 401 or 403 → api-invalid-key.spec.ts
• PATCH /api/v1/flows/{id} with wrong token does not update the flow → api-invalid-key.spec.ts
• rejects an expired API key with 403 and accepts a valid one with 200 → api-key-expiry-enforcement.spec.ts
• evaluates the expiry boundary in UTC, not shifted by the viewer offset → api-key-expiry-enforcement.spec.ts
• returns 200 with array → api-monitor-messages.spec.ts
• without auth returns 401 or 403 → api-monitor-messages.spec.ts
• filtered by session_id returns only matching messages → api-monitor-messages.spec.ts
• filtered by flow_id returns only matching messages → api-monitor-messages.spec.ts
• combined session_id and flow_id filters return 200 → api-monitor-messages.spec.ts
• messages contain required fields when not empty → api-monitor-messages.spec.ts
• executes flow with input_value and returns outputs → api-run-flow.spec.ts
• executes flow with custom session_id and persists messages under it → api-run-flow.spec.ts
• returns 404 for non-existent flow ID → api-run-flow.spec.ts
• tweaks override a component field at runtime → api-run-with-tweaks.spec.ts
• empty tweaks object is a no-op and leaves the flow default in effect → api-run-with-tweaks.spec.ts
• tweaks referencing a non-existent component are silently ignored → api-run-with-tweaks.spec.ts
• GET /api/v1/version returns 200 with a non-empty version string → api-version.spec.ts
• GET /api/v1/version reports the Langflow package and main_version → api-version.spec.ts
• GET /api/v1/version response has correct content-type → api-version.spec.ts
• GET /api/v1/version responds within 5 seconds → api-version.spec.ts
• POST /api/v1/version returns 405 Method Not Allowed → api-version.spec.ts

core-components/

• renders on canvas with default fields and handles → agent-component-regression.spec.ts
• system prompt accepts input and persists across flow reload → agent-component-regression.spec.ts
• model dropdown exposes manage-model-providers and lists configured models → agent-component-regression.spec.ts
• selecting a different-provider model swaps the canvas provider icon → agent-component-regression.spec.ts
• API Request component — renders on canvas with correct output and URL handles → api-request-component-regression.spec.ts
• API Request component — inspector fields accept configured values → api-request-component-regression.spec.ts
• API Request component — invalid URL is accepted by field and run shows error notification → api-request-component-regression.spec.ts
• API Request component — GET request returns 200 and output Data contains all required fields → api-request-component-regression.spec.ts
• API Request component — POST method executes POST verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — PUT method executes PUT verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — PATCH method executes PATCH verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — DELETE method executes DELETE verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — non-2xx HTTP response propagates status_code without crashing → api-request-component-regression.spec.ts
• API Request component — query parameters embedded in URL are sent and echoed → api-request-component-regression.spec.ts
• API Request component — inspector headers table accepts key + value cell entries → api-request-component-regression.spec.ts
• API Request component — cURL tab switches mode and field accepts a cURL command → api-request-component-regression.spec.ts
• API Request component — cURL mode parses command, auto-fills URL, executes GET and returns 200 → api-request-component-regression.spec.ts
• API Request component — body table accepts key + value cell entries when method is POST → api-request-component-regression.spec.ts
• API Request component — flow state persists in database after autosave (URL, method, headers) → api-request-component-regression.spec.ts
• Show Beta Components toggle controls visibility of beta components in the sidebar → beta-components-toggle-regression.spec.ts
• Chat Input — toggling showfiles exposes the Files inspector field → chat-input-files-field-regression.spec.ts
• Chat Input — uploading via the inspector populates the Files field → chat-input-files-field-regression.spec.ts
• Chat Input → Chat Output — inspector-attached file is rendered in the Playground message → chat-input-files-field-regression.spec.ts
• Chat Input — clicking the dismiss button on the Files field clears the value → chat-input-files-field-regression.spec.ts
• Chat Input component — renders on canvas with Message output handle and Input Text field → chat-input-output-component-regression.spec.ts
• Chat Output component — renders on canvas with Inputs handle and run button → chat-input-output-component-regression.spec.ts
• Chat Input → Chat Output connection is accepted on canvas (Message ↔ Message) → chat-input-output-component-regression.spec.ts
• Chat Input → Chat Output — Input Text value propagates to ChatOutput on run → chat-input-output-component-regression.spec.ts
• Chat Input — sender_name override is reflected in the Playground chat message → chat-input-output-component-regression.spec.ts
• Chat Input/Output — default sender_name is 'User' on input and 'AI' on output → chat-input-output-component-regression.spec.ts
• breaking-change outdated components alert with a Review action, not a silent Update → component-breaking-change-alert.spec.ts
• reviewing a single breaking change warns about disconnection and defaults to a backup → component-breaking-change-alert.spec.ts
• Review All flags every outdated component as breaking and pre-selects none → component-breaking-change-alert.spec.ts
• Should delete a single component with the Backspace key → componentDelete.spec.ts
• Should delete a single component via the node options menu → componentDelete.spec.ts
• Should delete multiple selected components with a marquee selection → componentDelete.spec.ts
• user can add components by hovering and clicking the plus icon → componentHoverAdd.spec.ts
• custom component code button should be pink when adding custom component → customComponentAdd.spec.ts
• Data Operations Text mode returns the Case Conversion result as a Message → data-operations-component.spec.ts
• Data Operations Word Count switches the Text-mode output to JSON and counts the text → data-operations-component.spec.ts
• Data Operations JSON mode selects a single key from an upstream JSON output → data-operations-component.spec.ts
• Data Operations Table mode filters the rows of an upstream Table output → data-operations-component.spec.ts
• All three legacy operations components name Data Operations as their replacement → data-operations-legacy-link.spec.ts
• The legacy banner link filters the sidebar to Data Operations → data-operations-legacy-link.spec.ts
• Searching a legacy operations name surfaces Data Operations with legacy components hidden → data-operations-legacy-link.spec.ts
• A legacy operations component still builds and returns its result → data-operations-legacy-link.spec.ts
• two API Request nodes expose the same field without duplicating its DOM id → duplicate-dom-ids-regression.spec.ts
• two Agent nodes expose the same field without duplicating its DOM id → duplicate-dom-ids-regression.spec.ts
• user can edit a URL tool action in Tool Mode and the edits persist → edit-tools.spec.ts
• the system must delete the handles from advanced fields when the code is updated → general-bugs-delete-handle-advanced-input.spec.ts
• any changes on the node must be saved on user interaction → general-bugs-save-changes-on-node.spec.ts
• Human Input renders the default Approve and Reject branch handles when added to the canvas → human-input-node-config.spec.ts
• adding a custom User Action creates its branch handle without a reload → human-input-node-config.spec.ts
• the configured branch handles persist after save and reload → human-input-node-config.spec.ts
• If-Else routes matching input through the True branch and skips the False branch → if-else-component-regression.spec.ts
• If-Else routes non-matching input through the False branch and skips the True branch → if-else-component-regression.spec.ts
• If-Else operator=contains routes a substring match through the True branch → if-else-component-regression.spec.ts
• If-Else operator=regex routes a valid pattern match through the True branch → if-else-component-regression.spec.ts
• If-Else operator=regex hides the case_sensitive advanced field → if-else-component-regression.spec.ts
• If-Else case_sensitive defaults to ON — mixed-case inputs route to the False branch → if-else-component-regression.spec.ts
• If-Else with case_sensitive=OFF treats mixed-case inputs as a match (True branch) → if-else-component-regression.spec.ts
• If-Else operator=greater than routes a numeric match (10 > 5) through the True branch → if-else-component-regression.spec.ts
• If-Else operator=less than routes a numeric match (2.5 < 10) through the True branch → if-else-component-regression.spec.ts
• If-Else operator=less than or equal routes an equal-operands match (5 <= 5) through the True branch → if-else-component-regression.spec.ts
• If-Else operator=greater than or equal routes an equal-operands match (5 >= 5) through the True branch → if-else-component-regression.spec.ts
• Show Legacy Components toggle controls visibility of legacy components in the sidebar → legacy-components-toggle-regression.spec.ts
• Loop component — renders correctly with all handles and output inspection buttons → loop-component-regression.spec.ts
• Loop component — run without connections shows build failed notification → loop-component-regression.spec.ts
• Loop component — stops after exhausting input DataFrame and emits aggregated done → loop-component-regression.spec.ts
• box-selecting two connected non-IO components and clicking Group collapses them into a single Group node → nested-grouping-regression.spec.ts
• ungrouping a Group node restores the original components and the edge between them → nested-grouping-regression.spec.ts
• importing a flow with outdated components raises the flow-level outdated notification → outdated-component-notification.spec.ts
• the outdated-notification count matches the per-node update indicators → outdated-component-notification.spec.ts
• text input field edit persists → parameters-panel-field-types.spec.ts
• dropdown field edit persists → parameters-panel-field-types.spec.ts
• textarea field edit persists → parameters-panel-field-types.spec.ts
• int field edit persists → parameters-panel-field-types.spec.ts
• tab field edit persists → parameters-panel-field-types.spec.ts
• toggle field edit persists → parameters-panel-field-types.spec.ts
• float field edit persists → parameters-panel-field-types.spec.ts
• slider field edit persists → parameters-panel-field-types.spec.ts
• code field edit persists → parameters-panel-field-types.spec.ts
• table field edit persists → parameters-panel-field-types.spec.ts
• key-pair field edit persists → parameters-panel-field-types.spec.ts
• input list field edit persists → parameters-panel-field-types.spec.ts
• Prompt Template component — renders on canvas with output handle → prompt-template-component-regression.spec.ts
• Prompt Template component — variables in curly braces generate dynamic input handles → prompt-template-component-regression.spec.ts
• Prompt Template component — removing a variable removes its input handle → prompt-template-component-regression.spec.ts
• Prompt Template component — replacing a variable updates handles accordingly → prompt-template-component-regression.spec.ts
• Prompt Template component — clearing the template removes all dynamic handles → prompt-template-component-regression.spec.ts
• Prompt Template component — modal edits persist in UI and in saved flow → prompt-template-component-regression.spec.ts
• Prompt Template — use_double_brackets toggle is exposed in the InspectionPanel with its upstream display name → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — default toggle state is OFF; f-string mode extracts {var} and treats {{var}} as literal → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — enabling toggle switches parser to mustache mode; {{var}} creates handle and {var} is ignored → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — disabling toggle reverts to f-string mode and variables are re-extracted under the new parser → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — use_double_brackets value persists in the autosaved flow → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — mustache {{ var }} (spaces inside braces) is rejected with an error toast and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — mustache {{var.attr}} (dot notation) is rejected with an error toast and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — mustache {{#section}}{{/section}} is rejected with the complex-syntax message and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — mustache {{{var}}} (triple braces) is rejected with the complex-syntax message and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — {var.attr} (dot notation) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {var name} (space inside identifier) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {var,name} (comma inside identifier) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {1var} (leading digit) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {} (empty braces) is accepted by the parser and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — repeating the same variable produces exactly one handle (deduplication contract) → prompt-template-invalid-patterns-regression.spec.ts
• saving a canvas component as a template makes it reusable from the sidebar → saveComponents.spec.ts
• should allow only one Chat Input on the canvas → singleton-components.spec.ts
• should not allow adding a Webhook while a Chat Input is on the canvas → singleton-components.spec.ts
• should not allow duplicating a Chat Input → singleton-components.spec.ts
• should not allow copying and pasting a Chat Input → singleton-components.spec.ts
• should allow only one Webhook on the canvas → singleton-components.spec.ts
• should not allow adding a Chat Input while a Webhook is on the canvas → singleton-components.spec.ts
• should not allow duplicating a Webhook → singleton-components.spec.ts
• should not allow copying and pasting a Webhook → singleton-components.spec.ts
• a component in Tool Mode can be grouped with its Agent consumer → tool-mode-group.spec.ts
• User should be able to use components as tool → tool-mode.spec.ts
• applying a single component update refreshes it, decrements the outdated count, and creates a backup → update-component-action.spec.ts
• user should be able to see errors on popups when raise an error → validate-raise-errors-components.spec.ts
• Webhook component — HTTP POST accepts JSON and plain-text bodies returning 202 → webhook-component-regression.spec.ts
• Webhook component — flow is saved to database and contains the Webhook node → webhook-component-regression.spec.ts
• Webhook component — cURL command in inspector shows valid POST URL with flow ID → webhook-component-regression.spec.ts
• Webhook component — empty data field returns empty Data object → webhook-component-regression.spec.ts
• Webhook component — endpoint field renders the actual webhook URL → webhook-component-regression.spec.ts
• Webhook component — copy button copies the endpoint URL to clipboard → webhook-component-regression.spec.ts
• Webhook component — POST to non-existent flow name returns 404 → webhook-component-regression.spec.ts
• Webhook component — valid JSON payload is propagated as structured Data output → webhook-component-regression.spec.ts
• Webhook component — invalid JSON payload is encapsulated in {payload: ...} → webhook-component-regression.spec.ts
• GET /api/v1/monitor/messages returns 200 with array response → webhook-component-regression.spec.ts

core-functionality/a2a/

• the Internal dropdown lists a locally published agent and calling it runs that flow → a2a-client-agent-internal.spec.ts
• published agent flow serves a spec-valid card → a2a-server-agent-card.spec.ts
• card overrides change exactly what the card advertises → a2a-server-agent-card.spec.ts
• card is 404 while the flow is not published → a2a-server-agent-card.spec.ts
• card is 404 for an unknown flow id → a2a-server-agent-card.spec.ts
• a flow without chat input and output cannot be published → a2a-server-agent-tab-publish.spec.ts
• publishing from the Agent tab serves a card at the advertised URL → a2a-server-agent-tab-publish.spec.ts
• the card editor changes what the API serves → a2a-server-agent-tab-publish.spec.ts
• the Try it panel round-trips a sentinel over the published endpoint → a2a-server-agent-tab-try-it.spec.ts
• the api-key gate follows the project the flow lives in → a2a-server-auth-apikey.spec.ts
• discovery lists only agent-typed, A2A-enabled flows → a2a-server-discovery.spec.ts
• unpublishing removes the flow from discovery → a2a-server-discovery.spec.ts
• message/send runs the flow and echoes the sentinel back → a2a-server-jsonrpc-message-send.spec.ts
• each call produces its own task → a2a-server-jsonrpc-message-send.spec.ts
• protocol errors come back as JSON-RPC errors over HTTP 200 → a2a-server-jsonrpc-message-send.spec.ts
• a conversation keeps its thread only while the caller quotes the contextId → a2a-server-multi-turn-context.spec.ts
• a task can be read back and refuses a cancel it cannot honour → a2a-server-tasks-lifecycle.spec.ts
• a task id is invisible to another flow → a2a-server-tasks-lifecycle.spec.ts
• cancelling a running task moves it to canceled → a2a-server-tasks-lifecycle.spec.ts

core-functionality/auth/

• logout must redirect user to login page → logout-flow.spec.ts
• after logout, navigating to root must redirect to login → logout-flow.spec.ts
• after logout, reload must stay on login page → logout-flow.spec.ts

core-functionality/knowledge-ingestion-management/

• upload a file through the Files page → file-types-upload.spec.ts
• should navigate to Files page and expose upload affordances → files-page.spec.ts
• should upload file using upload button → files-page.spec.ts
• should upload file using drag and drop → files-page.spec.ts
• should upload multiple files with different types → files-page.spec.ts
• should search uploaded files → files-page.spec.ts
• should handle bulk actions for multiple files → files-page.spec.ts
• user should not be able to upload a file larger than the limit → limit-file-size-upload.spec.ts
• Full RAG pipeline grounds the model answer on the retrieved chunk → rag-pipeline.spec.ts
• Split Text splits an ingested document into the expected number of chunks → split-text-chunking.spec.ts
• upload a file through the Read File component and read its content → upload-via-component.spec.ts
• Knowledge Base indexes the ingested document chunks (available for query) → vector-store-index-query.spec.ts
• Knowledge Base query returns the relevant chunk for the prompt → vector-store-index-query.spec.ts

core-functionality/llm-agents/

• agent interaction suite → agent-component-regression.spec.ts
• agent stop button must halt execution mid-run → agent-component-regression.spec.ts
• Agent settings survive save and reopen → agent-config-persistence.spec.ts
• context-scoped retrieval returns all turns of the context and not the untagged control → agent-context-id-continuity.spec.ts
• agent run persists every session message tagged with the custom context_id → agent-context-id-continuity.spec.ts
• mirrored context-scoped retrievals return only their own context's messages → agent-context-id-isolation.spec.ts
• switching the agent's context_id re-tags new turns without touching previous ones → agent-context-id-isolation.spec.ts
• toggle ON (default): agent's date tool returns today's date → agent-current-date-tool.spec.ts
• toggle OFF: the date tool is removed from the agent's toolkit → agent-current-date-tool.spec.ts
• model refusal does not crash the component → agent-empty-refusal-response.spec.ts
• empty response does not crash the component → agent-empty-refusal-response.spec.ts
• input via ChatInput handle drives the agent response → agent-input-sources.spec.ts
• input via the Agent's direct field drives the agent response → agent-input-sources.spec.ts
• causal control — a high max iterations does not hit the limit → agent-max-iterations.spec.ts
• max_tokens=50 caps the response's output tokens → agent-max-tokens.spec.ts
• causal control — unset max_tokens generates freely → agent-max-tokens.spec.ts
• selecting 'Connect other models' clears the previously selected model → agent-model-connection-isolation.spec.ts
• agent selects the URL tool for a fetch prompt → agent-multi-tool-selection.spec.ts
• agent selects the Web Search tool for a search prompt → agent-multi-tool-selection.spec.ts
• image via input handle is described by the agent → agent-multimodal-image-input.spec.ts
• negative control — no image, no image-specific description → agent-multimodal-image-input.spec.ts
• a small n_messages truncates retrieval to the most recent messages → agent-n-messages-limit.spec.ts
• causal control — a large n_messages retrieves the full seeded history → agent-n-messages-limit.spec.ts
• output_schema fields come back as typed JSON keys on the structured response → agent-structured-output.spec.ts
• a multiple (As List) schema row returns an array of the row's type → agent-structured-output.spec.ts
• Agent Instructions are respected in the model response → agent-system-prompt.spec.ts
• negative control — sentinel is absent without the instruction → agent-system-prompt.spec.ts
• agent handles a tool error and continues execution → agent-tool-error-handling.spec.ts
• an invalid tool name blocks execution with a clear message → agent-tool-name-validation.spec.ts
• causal control — a valid custom tool name executes normally → agent-tool-name-validation.spec.ts
• user must be able to send images in the playground with the agent component → general-bugs-agent-images-playground.spec.ts
• language model must respond with OpenAI provider → language-model-regression.spec.ts
• language model must respond with Google provider → language-model-regression.spec.ts
• language model provider switch from OpenAI to Google must persist → language-model-regression.spec.ts
• model provider dialog opens from the Language Model node → language-model-regression.spec.ts
• playground shows error when LLM run endpoint returns 500 (mocked invalid API key) → llm-invalid-api-key-ui.spec.ts
• playground input remains usable after API error (mocked) → llm-invalid-api-key-ui.spec.ts
• memory chatbot template loads with correct node structure → memory-history-regression.spec.ts
• message history context retention suite → memory-history-regression.spec.ts
• session isolation: new session has no context from previous session → memory-history-regression.spec.ts
• OpenAI provider is listed in Model Providers settings → model-provider-api-key.spec.ts
• Anthropic provider is listed in Model Providers settings → model-provider-api-key.spec.ts
• a configured provider exposes the key edit surface (Replace, no raw input) → model-provider-api-key.spec.ts
• page opens with its description and the available provider count → model-provider-modal-actions.spec.ts
• an invalid API key is rejected and does not enable the provider → model-provider-modal-actions.spec.ts
• selecting another provider switches the visible detail panel → model-provider-modal-actions.spec.ts
• model toggle changes immediately and persists across reopen → model-provider-model-toggle.spec.ts
• disabling a model removes it from a component model dropdown → model-provider-model-toggle.spec.ts
• the Language Model node renders its model selector → modelInputComponent.spec.ts
• opening the model dropdown lists model options → modelInputComponent.spec.ts
• the model dropdown exposes the Manage Model Providers entry → modelInputComponent.spec.ts
• the trigger shows the selected model name → modelInputComponent.spec.ts
• provider list renders with the known providers → modelProviderModal.spec.ts
• selecting a provider opens its API key configuration detail → modelProviderModal.spec.ts
• a configured provider shows its model selection panel → modelProviderModal.spec.ts
• should display error message when using invalid authentication for provider → provider-invalid-auth-error.spec.ts
• a provider credential variable can be removed through the Global Variables UI → remove-provider-api-key.spec.ts
• DELETE /api/v1/variables/{id} removes a provider API key variable → remove-provider-api-key.spec.ts

core-functionality/model-provider/

• Anthropic API key is configured via Settings → Model Providers → anthropic-provider.spec.ts
• configured Anthropic selects a Claude model in the Agent and executes the flow → anthropic-provider.spec.ts
• switches between Claude model families (Haiku → Sonnet → Opus) → anthropic-provider.spec.ts
• Azure AI Foundry is offered with a two-variable form and a Foundry-only deployment surface → azure-ai-foundry-provider-setup.spec.ts
• an unconfigured Azure AI Foundry panel is read-only: no enable toggle, no add-deployment control → azure-ai-foundry-provider-setup.spec.ts
• credentials that do not validate are rejected and nothing is persisted → azure-ai-foundry-provider-setup.spec.ts
• a portal deployment name absent from every catalog is accepted and rendered → azure-ai-foundry-provider-setup.spec.ts
• real credentials configure the provider and enable a portal deployment through the UI → azure-ai-foundry-provider-setup.spec.ts
• the configured deployment answers a real inference through the Language Model component → azure-ai-foundry-provider-setup.spec.ts
• Google API key is configured via Settings → Model Providers → google-provider.spec.ts
• configured Google selects a Gemini model in the Agent and executes the flow → google-provider.spec.ts
• Ollama base URL is configured via Settings → Model Providers → ollama-provider.spec.ts
• the provider is offered with two variables and a live-only, empty catalog → openai-compatible-provider-setup.spec.ts
• an unreachable base URL is rejected and nothing is persisted → openai-compatible-provider-setup.spec.ts
• a reachable endpoint with a bogus key is rejected as an authentication failure → openai-compatible-provider-setup.spec.ts
• a discovered model runs a flow through the OpenAI Compatible provider → openai-compatible-provider-setup.spec.ts
• saving a base URL and an API key through Settings persists BOTH variables → openai-compatible-provider-setup.spec.ts
• OpenAI API key is configured via Settings → Model Providers → openai-provider.spec.ts

core-functionality/observability-monitoring/

• a misconfigured flow surfaces an appropriate build-error message → flow-error-message.spec.ts
• Clearing traces for a flow whose trace has spans succeeds (cascade), leaving no traces behind → traces-delete-cascade.spec.ts
• DELETE /api/v1/monitor/traces returns 404 for an unknown flow_id → traces-delete.spec.ts
• DELETE /api/v1/monitor/traces?flow_id=... clears all traces, and a second DELETE on the empty owned flow still returns 204 → traces-delete.spec.ts
• GET /api/v1/monitor/traces/{trace_id} returns a populated tokenUsage + modelName on the LLM span → traces-detail-llm-span-populated.spec.ts
• GET /api/v1/monitor/traces/{trace_id} returns 404 for an unknown but well-formed UUID → traces-detail-single.spec.ts
• GET /api/v1/monitor/traces/{trace_id} returns the full TraceRead contract with a non-empty span tree → traces-detail-single.spec.ts
• GET /api/v1/monitor/transactions returns 200 with paginated result → traces-detail.spec.ts
• GET /api/v1/monitor/transactions filters by flow_id (UUID) → traces-detail.spec.ts
• transaction records contain required fields when not empty → traces-detail.spec.ts
• GET /api/v1/monitor/traces returns totalLatencyMs and totalTokens for a flow run → traces-latency-tokens.spec.ts
• Flow Activity page shows latency and token columns for the run → traces-latency-tokens.spec.ts
• Trace Details modal shows span tree and per-span latency → traces-latency-tokens.spec.ts
• GET /api/v1/monitor/traces?status=error returns only the failing trace; rejects unknown values → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?status=ok returns only the successful trace → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?start_time pins the >= lower bound → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?query= filters by trace name (incl. 50-char sanitize cap) → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?session_id filters by the session passed at run time → traces-list-filters.spec.ts
• should be able to see and interact with Traces → traces.spec.ts

core-functionality/playground/

• approving a Human Input pause routes only the approved branch → human-input-pause-resume.spec.ts
• rejecting a Human Input pause routes only the reject branch → human-input-pause-resume.spec.ts
• copy button copies Chat Input output and toggles Check icon → output-modal-copy-button.spec.ts
• playground must show one compact preview per attached image when two images are attached → playground-attachments-management.spec.ts
• playground must keep the remaining preview when one of two attachments is removed → playground-attachments-management.spec.ts
• playground must render both attached images in the user message after sending → playground-attachments-management.spec.ts
• playground input must return to empty state after removing the only attachment → playground-attachments-management.spec.ts
• playground swap flow must send only the second image when the first is removed before attaching the second → playground-attachments-management.spec.ts
• selecting an individual session checkbox must reveal the bulk-delete-button → playground-bulk-delete.spec.ts
• select-all-checkbox must select all non-default sessions → playground-bulk-delete.spec.ts
• bulk-delete-button must remove all selected sessions from the sidebar → playground-bulk-delete.spec.ts
• clear chat on Default session must remove messages but keep the session → playground-clear-history.spec.ts
• deleting a user-created session must remove it and return to Default session → playground-clear-history.spec.ts
• send button stays enabled regardless of input content → playground-empty-message-send.spec.ts
• clearing the input after typing leaves the field empty → playground-empty-message-send.spec.ts
• playground opens in fullscreen with chat input visible → playground-fullscreen.spec.ts
• playground closes and reopens correctly from the flow editor → playground-fullscreen.spec.ts
• messages sent in playground must persist after closing and reopening → playground-history-persist.spec.ts
• playground opens with chat textarea pre-filled from ChatInput Input Text → playground-input-text-prefill.spec.ts
• creating a new session re-applies the Input Text pre-fill → playground-input-text-prefill.spec.ts
• pre-filled value is sent as the first message of the session → playground-input-text-prefill.spec.ts
• edit user message — hover reveals edit button and saved changes replace original text → playground-message-edit.spec.ts
• cancel message edit — original text is preserved → playground-message-edit.spec.ts
• message edited in playground is reflected in Session Logs → playground-message-edit.spec.ts
• message-logs-option must open the Session Logs modal for the active session → playground-message-logs.spec.ts
• selecting messages in the log table and deleting them must reduce the row count → playground-message-logs.spec.ts
• playground must show non-image preview tile (delete button, no ) in input area after attaching a .txt file → playground-non-image-attachment.spec.ts
• playground must render non-image attachment in user message (truncated filename + zero file-images) after sending a .txt → playground-non-image-attachment.spec.ts
• playground must render JSON Data output as a code block → playground-output-data.spec.ts
• playground must render DataFrame output as a markdown table → playground-output-data.spec.ts
• playground must show image compact preview in input area after attaching an image → playground-output-image.spec.ts
• playground must display uploaded image in user message after sending → playground-output-image.spec.ts
• Playground run is delivered over an SSE (text/event-stream) response → playground-response-streaming-sse.spec.ts
• clear-chat removes all messages from Default Session → playground-session-clear.spec.ts
• a session renamed in the playground is the session its messages are stored under → playground-session-id.spec.ts
• new-chat button must add a new session entry to the sidebar → playground-session-nav.spec.ts
• session selector sidebar must switch to the selected session → playground-session-nav.spec.ts
• rename option must not be available for the Default Session → playground-session-rename.spec.ts
• rename option must not be available for a session with no messages → playground-session-rename.spec.ts
• rename option must be available and functional for a session with messages → playground-session-rename.spec.ts
• Shareable playground URL is generated when publishing is enabled → playground-shareable-url.spec.ts
• user message must appear instantly in playground before AI responds → playground-ux.spec.ts
• playground must scroll to latest message after sending → playground-ux.spec.ts
• playground input field must be ready after flow responds → playground-ux.spec.ts
• User must be able to stop building from inside Playground → stop-button-playground.spec.ts

core-functionality/project-management/

• user should be able to select flows with different methods and perform bulk actions → bulk-actions.spec.ts
• user should be able to edit flow name and see it reflected in the main page listing → edit-flow-name.spec.ts
• flow settings enforce character limits and persist name & description → flowSettings.spec.ts
• creates, renames and deletes an empty project folder via the UI → folder-crud.spec.ts
• deleting a folder that contains a flow removes the flow with it → folder-crud.spec.ts
• deleting a folder should update the folder list immediately → folder-deletion-integrity.spec.ts
• deleting one folder should not affect other folders → folder-deletion-integrity.spec.ts
• creating a new folder after deletion should work correctly → folder-deletion-integrity.spec.ts
• getting-started progress increments as onboarding steps complete → user-progress-track.spec.ts

flow-functionality/

• API access modal opens from the Publish dropdown exposing the Python, JavaScript and cURL tabs → api-access-modal-regression.spec.ts
• API access modal switches the displayed snippet when changing language tabs → api-access-modal-regression.spec.ts
• API access modal embeds the current flow ID in the generated run endpoint URL → api-access-modal-regression.spec.ts
• API access modal closes cleanly via Escape and via the close button → api-access-modal-regression.spec.ts
• user should be able to manually save a flow when the auto_save is off → auto-save-off.spec.ts
• connecting two compatible components creates exactly one edge → canvas-connect-components.spec.ts
• connecting the same compatible pair twice does not duplicate the edge → canvas-connect-components.spec.ts
• a type-incompatible pair does not connect → canvas-connect-components.spec.ts
• clicking the same target handle twice does not create an edge → canvas-connect-components.spec.ts
• copy and paste ChatOutput component via Ctrl+C / Ctrl+V → canvas-copy-paste.spec.ts
• copy and paste Prompt Template (component with dynamic ports) via Ctrl+C / Ctrl+V → canvas-copy-paste.spec.ts
• clicking empty canvas area deselects a selected node → canvas-deselect-node.spec.ts
• pressing Escape deselects a selected node → canvas-deselect-node.spec.ts
• deleting an edge from its context menu removes it from the canvas and the flow → canvas-edge-reconnect.spec.ts
• an edge can be recreated after it is deleted → canvas-edge-reconnect.spec.ts
• dragging a component moves it on the canvas and persists the new position → canvas-move-node.spec.ts
• a Shift+drag marquee selects every component it encloses → canvas-multiselect.spec.ts
• deleting a box selection clears the selected components → canvas-multiselect.spec.ts
• deleting a sticky note from its options menu removes it everywhere → canvas-sticky-note-delete.spec.ts
• deleting a sticky note with Backspace removes it everywhere → canvas-sticky-note-delete.spec.ts
• deleting one of two sticky notes leaves the other in place → canvas-sticky-note-delete.spec.ts
• user can create a blank flow from the new-project modal → create-blank-flow.spec.ts
• user can create a flow from a starter template → create-flow-from-template.spec.ts
• user can copy a valid macOS/Linux curl command from the API access modal → curlApiGeneration.spec.ts
• user can duplicate a flow from the home page dropdown menu → duplicate-flow.spec.ts
• duplicate flow via API auto-suffixes the name on collision → duplicate-flow.spec.ts
• export flow to JSON triggers success toast and produces a valid file → export-import-flow.spec.ts
• imported JSON flow must load all components on canvas → export-import-flow.spec.ts
• import flow from JSON via upload button must load flow on canvas → export-import-flow.spec.ts
• 1 - runs the flow from the canvas terminal node → flow-execution-canvas.spec.ts
• 2 - the flow ran correctly: every node reached build success → flow-execution-canvas.spec.ts
• 3 - the chat input and chat output are visible in the Playground → flow-execution-canvas.spec.ts
• should lock and unlock a flow and verify UI changes → flow-lock.spec.ts
• should show correct lock/unlock icon in settings based on state → flow-lock.spec.ts
• flow can be renamed via the header edit → flow-rename-header.spec.ts
• flow name persists after rename via API PATCH and GET → flow-rename-header.spec.ts
• import invalid JSON must show error message → import-invalid-json.spec.ts
• import non-JSON file must show error message → import-invalid-json.spec.ts
• import JSON with missing data field must show error → import-invalid-json.spec.ts
• user must be able to lock a flow and it must be saved → lock-flow.spec.ts
• user can publish a flow and access it via shareable URL, then unpublish to revoke access → publish-flow.spec.ts
• publish flow via API toggles access_type between PUBLIC and PRIVATE → publish-flow.spec.ts
• user can copy a valid Python requests snippet from the API access modal → pythonApiGeneration.spec.ts
• flow state should be properly cleaned up between user sessions → user-flow-state-cleanup.spec.ts

mcp/client/

• Gemini invokes the echo MCP tool (regression for fixed upstream #440) → mcp-client-agent-gemini-tool-regression.spec.ts
• configures MCP server via JSON, selects echo tool, runs it, and verifies output → mcp-client-regression.spec.ts
• configures MCP server via HTTP form tab and verifies registration → mcp-client-regression.spec.ts
• selects get-sum tool, provides numeric inputs, and verifies sum in output → mcp-client-regression.spec.ts
• registering an already-existing MCP server returns 409 Conflict → mcp-server-registration-status-codes.spec.ts
• deleting a non-existent MCP server returns 404 Not Found → mcp-server-registration-status-codes.spec.ts

mcp/server/

• generated endpoint advertises the project and lists the enabled flow → mcp-server-protocol.spec.ts
• execute the exposed tool over the MCP protocol echoes the input → mcp-server-protocol.spec.ts
• resources/list surfaces the uploaded flow file as a resource → mcp-server-resources.spec.ts
• user must be able to see starter projects for mcp servers → mcp-server-starter-projects.spec.ts
• user must not be able to add duplicate mcp servers from starter projects → mcp-server-starter-projects.spec.ts
• user should be able to manage MCP server tools and configuration → mcp-server-tab.spec.ts
• user must be able to add and delete MCP server from sidebar → mcp-server.spec.ts
• STDIO MCP server fields should persist after saving and editing → mcp-server.spec.ts
• HTTP/SSE MCP server fields should persist after saving and editing → mcp-server.spec.ts
• mcp server tools should be refreshed when editing a server → mcp-server.spec.ts
• stdio command with an embedded argument is refused, and command plus args is accepted → mcp-server.spec.ts

ui-ux/

• select and delete a flow → actionsMainPage-shard-1.spec.ts
• serializes created_at/expires_at with UTC offset and no microseconds → api-keys-timezone-display.spec.ts
• renders API key timestamps in the viewer's local timezone → api-keys-timezone-display.spec.ts
• zoom in and zoom out step the canvas scale and clamp at the React Flow bounds → canvas-zoom-navigation.spec.ts
• Fit View centers every node inside the canvas viewport → canvas-zoom-navigation.spec.ts
• Fit View is reachable from the canvas controls toolbar → canvas-zoom-navigation.spec.ts
• wheel scroll navigates the canvas anchored at the pointer → canvas-zoom-navigation.spec.ts
• user can edit the text of an existing sticky note and the canvas reflects only the new text → edit-sticky-note-text.spec.ts
• executing flow with network error shows error feedback → execution-error-notification.spec.ts
• executing flow with server error shows error feedback → execution-error-notification.spec.ts
• user must see on handle click the possibility connections → filterSidebar.spec.ts
• create a Generic global variable from Settings page → global-variable-edit.spec.ts
• create a Generic type global variable → global-variables-crud.spec.ts
• delete a global variable removes it from the list → global-variables-crud.spec.ts
• Credential variable value is hidden from the variable list → global-variables-crud.spec.ts
• user can search and add components using keyboard shortcuts → keyboardComponentSearch.spec.ts
• LangflowShortcuts → langflowShortcuts.spec.ts
• the main menu lists every item, reports the running version and links out → main-menu-actions.spec.ts
• the main menu's Settings action navigates to the Settings page → main-menu-actions.spec.ts
• user must be able to minimize and expand a component → minimize.spec.ts
• User should be able to interact notifications tab → notifications.spec.ts
• right-clicking a component selects it and opens its options menu → right-click-dropdown.spec.ts
• an item picked from the right-click menu acts on that component → right-click-dropdown.spec.ts
• right-clicking the canvas background opens no menu and dismisses an open one → right-click-dropdown.spec.ts
• Settings General section loads and shows its header → settings-general-section.spec.ts
• Settings Messages section is accessible → settings-general-section.spec.ts
• Settings Shortcuts section is accessible and lists shortcuts → settings-general-section.spec.ts
• Settings > Messages displays sent messages in correct order with working filters → settings-message-history.spec.ts
• user can access Settings page from the profile menu → settings-navigation.spec.ts
• Settings page shows all main sections in sidebar navigation → settings-navigation.spec.ts
• Settings Shortcuts section lists keyboard shortcuts → settings-navigation.spec.ts
• Settings Model Providers section loads with provider configuration → settings-navigation.spec.ts
• dark and light mode toggle correctly updates the body class → settings-theme-toggle.spec.ts
• double-click on a sidebar component adds it to the canvas → sidebar-add-component.spec.ts
• dragging a sidebar component drops the node at the pointer → sidebar-add-component.spec.ts
• an added component arrives with its catalog default settings → sidebar-add-component.spec.ts
• searching by name lists the matching component and hides the others → sidebar-search-and-filter.spec.ts
• a query with no match shows the empty state and clearing restores the tree → sidebar-search-and-filter.spec.ts
• a provider query groups its components under the provider bundle → sidebar-search-and-filter.spec.ts
• category disclosures collapse and expand their component list → sidebar-search-and-filter.spec.ts
• adding a sticky note places it on the canvas and in the flow → sticky-notes.spec.ts
• changing a sticky note colour repaints it and persists the choice → sticky-notes.spec.ts
• resizing a sticky note grows it and persists the new size → sticky-notes.spec.ts

---

🔵 Phase 1 — Next Delivery

Validate ([-]) and create ([ ]) in the modules below. See details in Part II.

Module	Validate ([-])	Create ([ ])
api/flows/ — REST API	0	0
core-components/ — Component Config	3	0
core-components/ — Core Components	3	1
core-functionality/auth/	13	0
core-functionality/llm-agents/	5	2
core-functionality/model-provider/	2	0
core-functionality/playground/	3	1
mcp/client/	2	2
mcp/server/	1	1
ui-ux/ — Canvas	0	0

---

🟡 Phase 2 — Next Delivery

Remaining modules after Phase 1 completion. See details in Part II.

Module	Validate ([-])	Create ([ ])
core-functionality/observability-monitoring/	0	0
core-functionality/knowledge-ingestion/	0	0
flow-functionality/	1	0
core-functionality/project-management/	6	0
core-functionality/templates/	39	0
ui-ux/ — Settings	0	0