QA-CHECKLIST.md

Langflow — Regression Test Checklist

Repository: C:/QAx/langflow-playwright/langflow-e2e Tests: tests/tests-automations/regression/ Config: playwright.config.ts Last updated: 2026-08-06

---

How to use this checklist

• [x] → automated and validated — only assigned when the underlying Playwright test carries the @stable tag
• [-] → automated, needs validation (test exists but is not yet @stable, or the entry refers to a Page/Helper rather than a test)
• [ ] (empty) → needs automation
• [~] → partially covered
• [!] → covered but flaky / unstable

---

---

PART I — PAGES & HELPERS

---

Pages

• [-] SimpleAgentTemplatePage — loads Simple Agent template with configurable provider and model → pages/SimpleAgentTemplatePage.ts
• [-] SettingsPage — navigation to the settings page via user menu → pages/SettingsPage.ts
• [-] Component sidebar — component navigation bar with searchable parameter support → covered by ui-ux/sidebar-search-and-filter.spec.ts, ui-ux/keyboardComponentSearch.spec.ts, ui-ux/sidebar-add-component.spec.ts (#820: no dedicated nav spec needed — exercised across the sidebar suite; #937 consolidated sidebar-provider-count / sidebar-category-filter / sidebar-filter-by-category into sidebar-search-and-filter)
• [-] Model Provider — navigation to the model provider management tab → covered by ui-ux/settings-navigation.spec.ts ("Settings Model Providers section loads") (#820)
• [-] API Keys — navigation to the API keys / global variables tab → covered by ui-ux/userSettings.spec.ts (API Keys), ui-ux/global-variable-edit.spec.ts + ui-ux/global-variables-crud.spec.ts (global variables) (#820)
• [-] Templates — navigation to the template selection tab (Starter Projects) → covered by core-functionality/templates/starter-projects.spec.ts, flow-functionality/create-flow-from-template.spec.ts (#820)
• Import Flow — navigation to import a flow via JSON → flow-functionality/export-import-flow.spec.ts (also flow-functionality/import-invalid-json.spec.ts; see §12.4) (#820)
• Delete Flow — navigation to delete a flow → ui-ux/actionsMainPage-shard-1.spec.ts ("select and delete a flow"); bulk via core-functionality/project-management/bulk-actions.spec.ts (#820)
• [-] MCP Config — navigation to configure MCP Server → covered by mcp/server/mcp-server-tab.spec.ts, mcp/server/mcp-server.spec.ts, core-components/configure-mcp-and-custom-component.spec.ts (#820)

---

Helpers

Provider Setup

• [-] OpenAI Provider Setup → helpers/provider-setup/setup-openai.ts
• [-] Anthropic Provider Setup → helpers/provider-setup/setup-anthropic.ts
• [-] Google Generative AI Provider Setup → helpers/provider-setup/setup-google.ts
• [-] Provider Map (providerSetupMap) — central registration point → helpers/provider-setup/index.ts
• [-] Provider validation via API (credit, valid key) → helpers/provider-setup/collect-models.ts
• [-] Provider build-axis probe (registry key present + component instantiates) → helpers/provider-setup/probe-component-buildable.ts
• [-] Collection of available models via UI (Settings → Model Providers) → helpers/provider-setup/collect-models.ts
• [-] providers.json — status of each provider (active/inactive + reason) → data/providers.json
• [-] models.json — list of models per provider → data/models.json

Flows

• [-] Load Simple Agent with variable provider and model → pages/SimpleAgentTemplatePage.ts
• [-] Load Simple Agent with OpenAI (wrapper) → helpers/flows/load-simple-agent-with-openai.ts

To implement

• [-] Configure an MCP → helpers/mcp/configure-mcp-server.ts
• [-] Configure a Custom Component → helpers/flows/configure-custom-component.ts
• Delete a component → helpers/flows/delete-component.ts
• Run a flow → helpers/flows/run-flow.ts
• [-] Pause a flow → flow-functionality/stop-building.spec.ts
• Send a chat input → flow-functionality/flow-execution-canvas.spec.ts
• Verify the chat output → flow-functionality/flow-execution-canvas.spec.ts

---

---

PART II — TEST AUTOMATION COVERAGE

Organized according to tests/tests-automations/regression/

---

api/ — REST API

api/flows/ — REST API

1.1 Health Check

• GET /health_check → status 200, db ok → api-health-check.spec.ts
• GET /api/v1/version → returns version, main_version, package → api-version.spec.ts

1.2 Flow CRUD via API

• POST /api/v1/flows/ → creates flow, returns ID → api/flows/api-flows-crud.spec.ts
• GET /api/v1/flows/ → lists user flows → api/flows/api-flows-crud.spec.ts
• GET /api/v1/flows/{id} → returns flow by ID → api/flows/api-flows-crud.spec.ts
• PATCH /api/v1/flows/{id} → updates name/description → api/flows/api-flows-crud.spec.ts
• DELETE /api/v1/flows/{id} → removes flow, returns 200 → api/flows/api-flows-crud.spec.ts
• GET /api/v1/flows/{id} after DELETE → should return 404 → api/flows/api-flows-crud.spec.ts

1.3 Flow Execution via API

• POST /api/v1/run/{flow_id} with input_value → returns response → api/flows/api-run-flow.spec.ts
• POST with tweaks → parameters override flow configuration → api/flows/api-run-with-tweaks.spec.ts
• POST with custom session_id → api/flows/api-run-flow.spec.ts
• POST with custom session_id persists messages retrievable via GET /api/v1/monitor/messages?session_id → api/flows/api-run-flow.spec.ts
• POST with input_type: "chat" and output_type: "chat" → api/flows/api-run-flow.spec.ts
• POST with invalid API key → returns 401/403 → api-invalid-key.spec.ts
• POST to non-existent flow → returns 404 → api/flows/api-run-flow.spec.ts

1.3.1 Folder (Project) CRUD via API

• POST /api/v1/projects/ → creates folder, returns id and name → api/flows/api-folders-crud.spec.ts
• GET /api/v1/projects/ → lists folders including the created one → api/flows/api-folders-crud.spec.ts
• [!] DELETE /api/v1/projects/{id} → returns 204 and the folder leaves the listing — quarantined (#965): under concurrent writes the endpoint answers 500 (sqlite3.OperationalError: database is locked) and the folder survives; measured 44% of deletes on 1.12.0.dev7 vs 6% on stable 1.10.3 at the same 2-client contention. Product defect filed as LE-2020; the 204 assertion is unchanged. The same ticket also covers PATCH /api/v1/flows/{id} (§12.5, #932) — one root cause, two endpoints → api/flows/api-folders-crud.spec.ts

1.4 Components via API

• GET /api/v1/all → lists all available components → api/flows/api-custom-component-creation.spec.ts
• POST /api/v1/custom_component → creates custom component → api/flows/api-custom-component-creation.spec.ts

1.5 Messages and Monitoring via API

• GET /api/v1/monitor/messages → returns 200 with array → api/flows/api-monitor-messages.spec.ts
• GET with session_id filter returns only messages from that session → api/flows/api-monitor-messages.spec.ts

1.6 Integration Code Generation

• Generate curl for API execution → flow-functionality/curlApiGeneration.spec.ts
• Generate Python code for integration → flow-functionality/pythonApiGeneration.spec.ts
• API access modal → flow-functionality/api-access-modal-regression.spec.ts

1.7 API Key Serialization & Expiry (PR #13471)

• GET /api/v1/api_key/ serializes created_at/expires_at as UTC ISO with +00:00 offset and no microseconds; null expires_at/last_used_at stay null → ui-ux/api-keys-timezone-display.spec.ts
• Expired API key is rejected on POST /api/v1/run/{id} with 403; valid key accepted with 200 → api/flows/api-key-expiry-enforcement.spec.ts
• Expiry boundary is evaluated in UTC, not shifted by viewer offset (±30 min UTC keys resolve correctly) → api/flows/api-key-expiry-enforcement.spec.ts

---

core-components/ — Component Configuration + Core Components

2. Component Configuration

2.1 Parameters Panel

• [-] Open component advanced options
• Edit text field (input) → core-components/parameters-panel-field-types.spec.ts
• Edit dropdown → core-components/parameters-panel-field-types.spec.ts
• Edit text area (textarea) → core-components/parameters-panel-field-types.spec.ts
• Edit code field → core-components/parameters-panel-field-types.spec.ts
• Edit float field → core-components/parameters-panel-field-types.spec.ts
• Edit int field → core-components/parameters-panel-field-types.spec.ts
• Edit toggle field → core-components/parameters-panel-field-types.spec.ts
• Edit key-pair list → core-components/parameters-panel-field-types.spec.ts
• Edit input list → core-components/parameters-panel-field-types.spec.ts
• Edit table input → core-components/parameters-panel-field-types.spec.ts
• Edit slider → core-components/parameters-panel-field-types.spec.ts
• Edit tab component → core-components/parameters-panel-field-types.spec.ts
• [-] Visibility toggle of a connected input is disabled (tooltip "Cannot change visibility of connected handles") and re-enables once the edge is deleted → flow-functionality/general-bugs-hidden-input-edges.spec.ts
• Two nodes on the canvas exposing the same field name render distinct DOM ids, while data-testid stays unscoped so both nodes remain selectable (LE-2037 / langflow#14312) → core-components/duplicate-dom-ids-regression.spec.ts

2.2 Tool Mode

• Enable Tool Mode on a component → core-components/tool-mode.spec.ts
• Group components in Tool Mode → core-components/tool-mode-group.spec.ts
• Edit tools (slug, description, requires-approval persistence) → core-components/edit-tools.spec.ts

2.3 Component Updates

• Outdated component notification → core-components/outdated-component-notification.spec.ts
• Update component action → core-components/update-component-action.spec.ts
• Update with breaking change — should alert user → core-components/component-breaking-change-alert.spec.ts
• Legacy component visible via configuration → core-components/legacy-components-toggle-regression.spec.ts
• Beta component visible via configuration → core-components/beta-components-toggle-regression.spec.ts
• Re-saving code removes handles from previously-toggled advanced fields → core-components/general-bugs-delete-handle-advanced-input.spec.ts

2.4 Code Editing

• Edit Python code of custom component — Check & Save clears the pulse-pink indicator → core-components/customComponentAdd.spec.ts
• Full custom component → core-components/full-custom-component.spec.ts
• [-] configureCustomComponent helper compiles code into a node with its declared interface → core-components/configure-mcp-and-custom-component.spec.ts

---

3. Core Components

3.1 Chat Input / Output

• ChatInput renders on canvas with Message output handle and Input Text field → core-components/chat-input-output-component-regression.spec.ts
• ChatOutput renders on canvas with Inputs handle and run button → core-components/chat-input-output-component-regression.spec.ts
• ChatInput → ChatOutput connection accepted (Message ↔ Message) → core-components/chat-input-output-component-regression.spec.ts
• Input Text propagates from ChatInput to ChatOutput on run → core-components/chat-input-output-component-regression.spec.ts
• Sender name override is reflected in the Playground chat message → core-components/chat-input-output-component-regression.spec.ts
• Default sender_name is "User" on input and "AI" on output → core-components/chat-input-output-component-regression.spec.ts
• Toggling showfiles exposes the Files inspector field on Chat Input → core-components/chat-input-files-field-regression.spec.ts
• Uploading a file via the Chat Input inspector populates the Files field → core-components/chat-input-files-field-regression.spec.ts
• Inspector-attached file is rendered in the Playground after running ChatInput → ChatOutput → core-components/chat-input-files-field-regression.spec.ts
• Dismiss button on the Files field clears the value → core-components/chat-input-files-field-regression.spec.ts
• Chat Input is a singleton — adding one removes both the Chat Input and Webhook + buttons from the sidebar (mutual exclusion) → core-components/singleton-components.spec.ts
• A value typed on a node (Chat Input as the host) is persisted by the debounced autosave and rehydrated after leaving and re-entering the flow — four consecutive edits, each gated on the server before the exit → core-components/general-bugs-save-changes-on-node.spec.ts
• Chat Input cannot be duplicated (Cmd/Ctrl+D) or copy/pasted (Cmd/Ctrl+C+V) — blocked with the "components were not pasted" toast → core-components/singleton-components.spec.ts
• [-] File on the advanced files field can be removed and re-uploaded; after running, the image and the user message render in the Playground → flow-functionality/general-bugs-shard-3836.spec.ts

3.2 Prompt Template

• Prompt Template renders on canvas with output handle → core-components/prompt-template-component-regression.spec.ts
• Variables in curly braces generate dynamic input handles → core-components/prompt-template-component-regression.spec.ts
• Removing a variable removes its input handle → core-components/prompt-template-component-regression.spec.ts
• Replacing a variable updates handles accordingly → core-components/prompt-template-component-regression.spec.ts
• Clearing the template removes all dynamic handles → core-components/prompt-template-component-regression.spec.ts
• Modal edits persist in UI and in saved flow → core-components/prompt-template-component-regression.spec.ts
• use_double_brackets toggle is exposed in the InspectionPanel with its upstream display name → core-components/prompt-template-double-brackets-regression.spec.ts
• Default toggle state is OFF; f-string mode extracts {var} and treats {{var}} as literal → core-components/prompt-template-double-brackets-regression.spec.ts
• Enabling toggle switches parser to mustache mode; {{var}} creates handle and {var} is ignored → core-components/prompt-template-double-brackets-regression.spec.ts
• Disabling toggle reverts to f-string mode and variables are re-extracted under the new parser → core-components/prompt-template-double-brackets-regression.spec.ts
• use_double_brackets value persists in the autosaved flow → core-components/prompt-template-double-brackets-regression.spec.ts
• f-string parser rejects {var.attr} (dot notation) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser rejects {var name} (space inside identifier) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser rejects {var,name} (comma inside identifier) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser rejects {1var} (leading digit) with an error toast and creates no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser accepts {} (empty braces) silently — no error, no handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• f-string parser deduplicates repeated variables — {name} and {name} yields exactly one handle → core-components/prompt-template-invalid-patterns-regression.spec.ts
• mustache parser rejects {{ var }} (spaces inside braces) with an error toast and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts
• mustache parser rejects {{var.attr}} (dot notation) with an error toast and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts
• mustache parser rejects {{#section}}{{/section}} with the complex-syntax message and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts
• mustache parser rejects {{{var}}} (triple braces) with the complex-syntax message and creates no handle → core-components/prompt-template-invalid-mustache-patterns-regression.spec.ts

3.3 API Request (HTTP)

• Renders on canvas with URL and API Response handles → core-components/api-request-component-regression.spec.ts
• Inspector fields accept URL and HTTP method values → core-components/api-request-component-regression.spec.ts
• Execute GET request and verify 200 status and output structure → core-components/api-request-component-regression.spec.ts
• Execute POST request and verify POST verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Execute PUT request and verify PUT verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Execute PATCH request and verify PATCH verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Execute DELETE request and verify DELETE verb is sent (status 200) → core-components/api-request-component-regression.spec.ts
• Non-2xx HTTP response (404) propagated as status_code without crash → core-components/api-request-component-regression.spec.ts
• Query parameters embedded in URL are sent and echoed in response → core-components/api-request-component-regression.spec.ts
• Invalid URL error shows notification with descriptive error message → core-components/api-request-component-regression.spec.ts
• Headers table accepts key + value cell entries via inspector → core-components/api-request-component-regression.spec.ts
• cURL tab switches mode and exposes the cURL input field → core-components/api-request-component-regression.spec.ts
• cURL parser auto-fills URL field and executes the GET, returning 200 → core-components/api-request-component-regression.spec.ts
• Body table accepts key + value cell entries when method is POST (body field is advanced=True and hidden by inspector while method is GET) → core-components/api-request-component-regression.spec.ts
• Flow state (URL, method, headers row) persists in database after autosave and rehydrates on reload → core-components/api-request-component-regression.spec.ts
• include_httpx_metadata=true adds the outgoing request headers as a top-level headers key in the output Data (advanced field, added to the node body via the inspector) → api/flows/api-component-regression.spec.ts
• Request timeout shorter than the endpoint's delay returns status_code 500 with an error field instead of raising (advanced field, added to the node body via the inspector) → api/flows/api-component-regression.spec.ts

3.4 Webhook

• POST aceita JSON e text/plain retornando 202 com status: "in progress" → core-components/webhook-component-regression.spec.ts
• Flow salvo no banco contém o nó Webhook com endpoint="BACKEND_URL" → core-components/webhook-component-regression.spec.ts (quarentena de 2½ meses liberada em #990: o teste lia o flow via page.evaluate(fetch) e tropeçava no defeito do interceptor de window.fetch do frontend; agora lê via request.get autenticado)
• Campo cURL no inspector mostra URL válida com flow ID e flags corretas (-X POST, Content-Type, -d) → core-components/webhook-component-regression.spec.ts
• Data field vazia retorna objeto Data vazio {} ao executar → core-components/webhook-component-regression.spec.ts
• Campo endpoint (str_endpoint) renderiza a URL real do webhook → core-components/webhook-component-regression.spec.ts
• Botão de cópia copia a URL correta para o clipboard e exibe toast "Endpoint URL copied" → core-components/webhook-component-regression.spec.ts
• POST para flow inexistente retorna 404 → core-components/webhook-component-regression.spec.ts
• GET /api/v1/monitor/messages retorna 200 com array → core-components/webhook-component-regression.spec.ts
• Payload JSON recebido é propagado corretamente como saída Data do componente → core-components/webhook-component-regression.spec.ts
• Payload inválido (não-JSON) é encapsulado em {"payload": "..."} na saída → core-components/webhook-component-regression.spec.ts
• Webhook is a singleton — adding one removes both the Webhook and Chat Input + buttons from the sidebar (mutual exclusion) → core-components/singleton-components.spec.ts
• Webhook cannot be duplicated (Cmd/Ctrl+D) or copy/pasted (Cmd/Ctrl+C+V) — blocked with the "components were not pasted" toast → core-components/singleton-components.spec.ts
• [-] Generated cURL includes the x-api-key header when webhook auth is enabled (mocked GET /api/v1/config, auto-login off) → core-components/general-bugs-component-webhook-api-key-display.spec.ts
• [-] Generated cURL omits x-api-key when webhook auth is disabled → core-components/general-bugs-component-webhook-api-key-display.spec.ts

3.5 Agent (Component)

• Agent component renders on canvas with title, handles and default fields → core-components/agent-component-regression.spec.ts
• System prompt accepts input and persists across flow reload → core-components/agent-component-regression.spec.ts
• Model dropdown exposes manage-model-providers and lists configured models → core-components/agent-component-regression.spec.ts
• Selecting a different-provider model swaps the canvas provider icon → core-components/agent-component-regression.spec.ts

3.6 Loop Component

• Loop component renders on canvas with title and run button → core-components/loop-component-regression.spec.ts
• Correct handles: inputs-left, item-left, item-right, done-right → core-components/loop-component-regression.spec.ts
• Output inspection buttons present for item and done → core-components/loop-component-regression.spec.ts
• Run without connections shows "Flow build failed" notification without crash → core-components/loop-component-regression.spec.ts
• Loop iterates over 2 ArXiv articles (Research Translation Loop template) and aggregates response in Playground → core-components/loop-component-regression.spec.ts
• Loop stops when exit condition is met → core-components/loop-component-regression.spec.ts

3.7 Nested / Grouping

• Nested component → core-components/nested-grouping-regression.spec.ts
• Enter and exit grouped component → core-components/nested-grouping-regression.spec.ts

3.8 If-Else Component

• operator=equals: matching input routes through True branch (False branch stays inactive) → core-components/if-else-component-regression.spec.ts
• operator=equals: non-matching input routes through False branch (True branch stays inactive) → core-components/if-else-component-regression.spec.ts
• operator=contains substring routing → core-components/if-else-component-regression.spec.ts
• operator=regex valid pattern routing → core-components/if-else-component-regression.spec.ts
• operator=regex hides case_sensitive field via update_build_config → core-components/if-else-component-regression.spec.ts
• case_sensitive ON (default) treats mixed case as no-match → core-components/if-else-component-regression.spec.ts
• case_sensitive OFF treats mixed case as a match → core-components/if-else-component-regression.spec.ts
• operator=greater than numeric routing → core-components/if-else-component-regression.spec.ts
• Other numeric operators (less than, less than or equal, greater than or equal) — share the same float(...) cast as greater than → core-components/if-else-component-regression.spec.ts
• max_iterations + default_route cycle break (not implementable as a standalone If-Else feedback loop on 1.12.x — Langflow forms graph cycles only via loop-aware target handles (from_loop_target_handle, target_handle.type is None) that LoopComponent ports provide; a feedback edge into the router's regular match_text field-input persists in the flow JSON but does not make the graph iterate (is_cyclic stays false, router runs once). conditional_router.py's cycle-break only fires when the router already sits inside a Loop-created cycle. Confirmed live on 1.12.0.dev3; product finding filed upstream; #891, follow-up of #822)

3.9 Human Input (HITL, 1.11.0)

• Human Input node config: default Approve/Reject branch handles, custom User Action creates a new handle, configured handles persist after save + reload → core-components/human-input-node-config.spec.ts

3.10 Data Operations (1.11.0)

• Data Operations component: unified JSON/Table/Text operations produce correct outputs per operation mode (Text→Message, Word Count→JSON override, JSON Select Keys, Table Filter) → core-components/data-operations-component.spec.ts
• Legacy operations components link/redirect to Data Operations (banner resolves the replacement, link filters the sidebar, old names still find the new component, legacy flows keep working) → core-components/data-operations-legacy-link.spec.ts

---

core-functionality/ — Core and Operational Logic

core-functionality/auth/ — Authentication and User Management

4.1 Login / Logout

• [-] Login with valid credentials
• [-] Login with invalid credentials — should display error message
• Logout — should redirect to login screen → core-functionality/auth/logout-flow.spec.ts
• [-] Auto-login enabled — should skip login screen
• [-] Auto-login disabled — should display login screen
• [-] Expired session — should redirect to login
• Session cleanup after logout — after logging out, navigating to / and reloading both stay on the login screen → core-functionality/auth/logout-flow.spec.ts

4.2 User Management (Admin)

• [-] Admin creates new user
• [-] Admin deactivates user
• [-] Admin activates inactive user
• [-] Admin renames user
• [-] Admin changes user password
• [-] Admin changes password — old password does not work after change
• [-] Isolation flow: user A cannot see user B's flows

4.3 Global Variables (API Keys)

• Create global variable
• [-] Use global variable in component (API key) → ui-ux/use-global-variable-in-component.spec.ts
• [!] Edit existing global variable — quarantined (#1235): clicking the variable's ag-grid row does not open the Update Variable modal, recurrent on the dailies of 2026-07-27 and 2026-08-03. Same surface and same shape as the provider-credential removal below (§7.5) — filed as one cause → ui-ux/global-variable-edit.spec.ts
• Delete global variable → ui-ux/global-variables-crud.spec.ts
• Create global variable of type "Generic" → ui-ux/global-variables-crud.spec.ts
• Credential variable value is hidden from the variable list → ui-ux/global-variables-crud.spec.ts
• Create global variable from Settings page → ui-ux/global-variable-edit.spec.ts

---

core-functionality/knowledge-ingestion-management/ — Upload, Processing and Vectors

5.1 File Upload

• Upload file via component → core-functionality/knowledge-ingestion-management/upload-via-component.spec.ts
• Upload files of different types (txt, pdf, json, py, wav) → core-functionality/knowledge-ingestion-management/file-types-upload.spec.ts
• File size limit → core-functionality/knowledge-ingestion-management/limit-file-size-upload.spec.ts
• File management page → core-functionality/knowledge-ingestion-management/files-page.spec.ts

5.2 Processing and Vectorization

• Split Text chunking of an ingested document → core-functionality/knowledge-ingestion-management/split-text-chunking.spec.ts
• Indexing in Vector Store — document available for query → core-functionality/knowledge-ingestion-management/vector-store-index-query.spec.ts
• Vector Store query returns relevant chunks for the prompt → core-functionality/knowledge-ingestion-management/vector-store-index-query.spec.ts
• Complete RAG pipeline (ingest → embed → store → retrieve → answer) → core-functionality/knowledge-ingestion-management/rag-pipeline.spec.ts

---

core-functionality/llm-agents/ — Agents and LLM Execution

⚠️ Tests in this section use SimpleAgentTemplatePage and are parameterized by model via models.json. Run npx playwright test tests/collect-models.spec.ts before executing these tests. See CLAUDE.md in this folder for the complete guide.

6.1 llm-agents/agent-component-regression.spec.ts — Agent Behavior Regression @stable

• Agent responds without connected tools
• Agent displays valid response and optionally reasoning steps
• Stop button interrupts agent execution
• Execution duration displayed after successful run
• Response displayed progressively in the Playground (streaming)
• Duration indicator displayed on canvas (node_duration_agent) after closing the playground
• Agent responds to multiple consecutive messages in the same session

6.2 Other execution tests

• [-] Agent displays reasoning steps in Playground → agent-reasoning-steps.spec.ts
• [-] Composio (tool integration for Agent) → composio.spec.ts
• Playground shows error when LLM run endpoint returns 500 (mocked invalid API key) → llm-agents/llm-invalid-api-key-ui.spec.ts
• Playground input remains usable after API error (mocked) → llm-agents/llm-invalid-api-key-ui.spec.ts
• [~] Agent stops when configured stop condition is reached → core-functionality/llm-agents/agent-max-iterations.spec.ts (max_iterations is the Agent's only configurable stop mechanism — no dedicated stop-condition field exists; see #824. Partial for the same reason as the bullet below: the stop itself is quarantined against #1264)
• [~] Agent stops when maximum number of iterations is reached → core-functionality/llm-agents/agent-max-iterations.spec.ts (partial: the stop assertion is quarantined (test.fixme, never @stable) against a live product failure — #1264. With max_iterations=1 on a task that needs several tool-calling iterations, the agent answers the task normally instead of returning Model call limits exceeded: run limit (1/1); the message renders, so this is a content failure, not a timeout. Reproduced on 1.12.0.dev18 locally, off CI load, on claude-haiku-4-5, claude-opus-5 and claude-opus-4-5, which rules out the mid-run backend wedge #1264's triage left open as a cover. The @stable causal control — a high limit finishes without the limit message — still runs, so what remains proven is that a high cap does not stop the agent, not that a low one does)
• Agent with multiple configured tools executes correctly → agent-multi-tool-selection.spec.ts
• Agent with configured timeout respects the limit (no product surface on 1.12.x — the Agent component exposes no timeout field: its inputs are max_iterations, max_tokens, n_messages, system_prompt, context_id, stream, and tools; Langflow's only configurable per-component timeouts live on the A2A Agent (timeout) and MCP tools (tool_execution_timeout), not the Agent, and the Language Model / chat models expose none either. The client/transport execution timeout that bounds any run is covered by ui-ux/execution-error-notification.spec.ts. Not automatable as written; #825, same class as #824)
• Connecting an external model in Agent drops the prior model selection (connection-mode isolation, prevents stale provider config) → llm-agents/agent-model-connection-isolation.spec.ts
• Flow with Agent saved and reopened → settings preserved → core-functionality/llm-agents/agent-config-persistence.spec.ts
• max_tokens truncates response as configured → llm-agents/agent-max-tokens.spec.ts (validated at token level via the Playground token-usage tooltip)

6.3 Memory and Context

• Memory Chatbot template loads with correct node and edge structure → llm-agents/memory-history-regression.spec.ts
• Message History retains context between messages in the same Playground session → llm-agents/memory-history-regression.spec.ts
• Session isolation: distinct session IDs have independent histories → llm-agents/memory-history-regression.spec.ts
• Without Message History, LLM does not retain context between messages → llm-agents/memory-history-regression.spec.ts
• n_messages parameter limits the number of retained messages → llm-agents/agent-n-messages-limit.spec.ts (bug reported fixed on 1.11.0.dev33 — parameter now respected; validated by deterministic message count)
• Agent uses custom context_id — continuity between session messages → agent-context-id-continuity.spec.ts
• Switching context_id isolates history between distinct sessions → agent-context-id-isolation.spec.ts

6.4 Tools and Integrations

• Agent with integrated external MCP tool executes action and returns result
• [-] Agent executes multiple tools in sequence → llm-agents/agent-multi-tool-selection.spec.ts (Test 3 — chained fetch→search, ordered tool_use assert; @stable gated on the clean baseline #818, per #827)
• Tool returns error — agent handles it and continues execution → core-functionality/llm-agents/agent-tool-error-handling.spec.ts
• Multiple connected tools — agent selects the correct one for each prompt → agent-multi-tool-selection.spec.ts
• Tool with invalid name — validation prevents execution with clear message → core-functionality/llm-agents/agent-tool-name-validation.spec.ts

6.5 Output and Reasoning

• [-] Inspect tools used by Agent in Playground → llm-agents/agent-tool-inspection.spec.ts (UI chip names the tool + persisted tool_use input/output; @stable gated on the clean baseline #818, per #827)
• Agent returns output in structured JSON format (output_schema) → agent-structured-output.spec.ts
• [-] Agent returns output in correctly rendered Markdown → llm-agents/agent-markdown-output.spec.ts
• Agent Instructions (system prompt) is respected in the model response → agent-system-prompt.spec.ts
• Input via direct field vs handle (ChatInput) — both work → core-functionality/llm-agents/agent-input-sources.spec.ts
• Empty response or model refusal — component does not crash → core-functionality/llm-agents/agent-empty-refusal-response.spec.ts
• Toggle add_current_date_tool works (enables/disables date tool) → agent-current-date-tool.spec.ts
• [~] handle_parsing_errors=False fails explicitly vs True auto-corrects → agent-parse-error-behavior.spec.ts (partially covered on 1.11: the field is present and togglable, but True/False are behaviorally identical — the field now only toggles ToolRetryMiddleware, and component-tool failures are converted to content by the hardcoded handle_tool_error=True before the middleware can observe them; the only live trigger (LLM-emitted malformed args) is non-deterministic, so the semantic difference is not deterministically testable — re-scope tracked in #496)
• Image passed via input handle is processed correctly → core-functionality/llm-agents/agent-multimodal-image-input.spec.ts
• Image attached in the Playground is processed by the Agent — the attachment renders in the user message and the reply describes it → core-functionality/llm-agents/general-bugs-agent-images-playground.spec.ts (@stable restored in #992 — the OpenAI quota that caused the #772 quarantine is back)

---

core-functionality/model-provider/ — Provider Management

⚠️ Provider configuration tests via Settings use SettingsPage. See helpers/provider-setup/ for the setup helpers of each provider.

7.1 Provider Collection and Validation

• Validate API keys of all providers via real call → collect-models.spec.ts
• Validate the running build can instantiate each provider's component (registry + build, not just the key) → collect-models.spec.ts
• Collect available models per provider via UI → collect-models.spec.ts
• Inactive providers appear as skipped in tests with reason → agent-component-regression.spec.ts
• Configure provider API key via Save Configuration (first setup) → collect-models.spec.ts
• Replace provider API key via Replace Configuration (existing key) → collect-models.spec.ts

7.2 OpenAI

• Configure OpenAI API key in Settings → Model Providers → core-functionality/model-provider/openai-provider.spec.ts
• Select GPT model in agent → core-functionality/model-provider/openai-provider.spec.ts
• Execute flow with OpenAI → core-functionality/model-provider/openai-provider.spec.ts
• Invalid API key error — display error message → core-functionality/llm-agents/provider-invalid-auth-error.spec.ts

7.3 Anthropic

• Configure Anthropic API key in Settings → Model Providers → core-functionality/model-provider/anthropic-provider.spec.ts
• Select Claude model in agent → core-functionality/model-provider/anthropic-provider.spec.ts
• Switch between Claude models (Sonnet, Haiku, Opus) → core-functionality/model-provider/anthropic-provider.spec.ts
• Invalid Anthropic API key error → core-functionality/llm-agents/provider-invalid-auth-error.spec.ts

7.4 Google Generative AI

• Configure Google API key in Settings → Model Providers → core-functionality/model-provider/google-provider.spec.ts
• Select Gemini model in agent → core-functionality/model-provider/google-provider.spec.ts
• Invalid Google API key error → core-functionality/llm-agents/provider-invalid-auth-error.spec.ts

7.5 Provider Management

• "Manage Model Providers" modal → llm-agents/modelProviderModal.spec.ts + llm-agents/model-provider-modal-actions.spec.ts
• Available provider count → llm-agents/model-provider-modal-actions.spec.ts
• Language Model component — configuration → llm-agents/language-model-regression.spec.ts
• Model Input component → llm-agents/modelInputComponent.spec.ts
• Add new provider via modal → llm-agents/model-provider-api-key.spec.ts (positive add validated via invalid-key rejection + Replace edit surface — a real re-add poisons a backend credential cache, see #505)
• Remove API key from existing provider — both paths validated. The UI path was quarantined (#1235) as a Global Variables permission-gate flake and is not one: the deletion already succeeded, and the spec then re-clicked the now-disabled header button through a phantom confirmation step. Fixed by asserting the header action is enabled and scoping the confirmation to a real dialog → llm-agents/remove-provider-api-key.spec.ts
• Per-model enable/disable toggle changes immediately and persists across reopen → llm-agents/model-provider-model-toggle.spec.ts
• Disabling a model in Settings removes it from a component model dropdown; re-enabling restores it → llm-agents/model-provider-model-toggle.spec.ts

7.6 Open-Source Providers

• Configure and execute flow with Ollama (local model) → model-provider/ollama-provider.spec.ts (both halves @stable again after #931: the lfx-ollama packaging gap that broke it on 07-23/24 is fixed upstream, a build-side bundle pre-flight now fails attributed instead of timing out blind for 30 s, the run-completion wait was rebuilt on the button-stop/button-send signal — the 07-15 failure mode — and the test model is derived from the instance instead of a hardcoded tag that could skip silently. Restored on 4 consecutive green manual.yml runs on 1.12.0.dev9, not on local evidence)
• [-] Configure and execute flow with Groq → model-provider/groq-provider.spec.ts (automated but not validatable on the tested image: the Groq component ships in the lfx-bundles distribution, which langflowai/langflow-nightly:latest does not install, so the spec's availability pre-flight skips it on every run — @stable removed, see #1039)
• [-] Configure and execute flow with Mistral → model-provider/mistral-provider.spec.ts (automated but not validatable on the tested image: same lfx-bundles packaging decision as Groq — @stable removed, see #1039)

7.7 Model Parameters (Agent)

• Maximum token count — response truncated as configured → llm-agents/agent-max-tokens.spec.ts
• [~] Maximum agent iterations → core-functionality/llm-agents/agent-max-iterations.spec.ts (partial — the stop assertion is quarantined against #1264; see §6.2 for the measurement)
• Use of custom context_id for memory isolation → agent-context-id-isolation.spec.ts
• Output formatting (JSON via output_schema, Markdown, plain text) → agent-structured-output.spec.ts

7.8 Unified Provider Setup — 1.11.0 additions

• [~] OpenAI Compatible as a first-class unified model provider: setup with base URL + key, models discovered, usable by a flow → core-functionality/model-provider/openai-compatible-provider-setup.spec.ts (6 tests, validated on 1.12.0.dev15 against https://api.openai.com/v1 — the endpoint the issue itself suggests, so no new CI secret: the base URL defaults there and the bearer is the existing OPENAI_API_KEY, overridable via OPENAI_COMPATIBLE_TEST_BASE_URL / _API_KEY / _MODEL. 5 are @stable: the two-variable form with its asymmetric required/optional Save gate; the live-only catalog — this is the first provider with no static rows at all, so unconfigured it contributes 0 models, asserted differentially against Azure AI Foundry's seed catalog in the same run; an unresolvable .invalid base URL and a real endpoint with a bogus key both rejected on the validate-provider body (HTTP 200 + valid:false, DNS vs Authentication failed for the OpenAI-compatible endpoint) with nothing persisted; live discovery returning exactly the endpoint's own /v1/models id set, twice over (once per model type); and one discovered id running a Basic Prompting flow to a sentinel reply through the provider-qualified OpenAI Compatible-<id>-option. Partial because the 6th — configuring the provider through Settings — is quarantined (test.fixme, no @stable) against a confirmed product defect: Save persists only the base URL, because the frontend fires the two POST /api/v1/variables/ writes concurrently and the primary (secret) key write is rejected 400 Invalid OpenAI-compatible base URL — validated against provider variables that do not yet include the base URL its sibling request is creating — with nothing surfaced in the UI. Measured 3/3 through the UI and isolated against the API on the same instance: sequential writes 201/201, concurrent 201/400. So an authenticated OpenAI-compatible endpoint cannot be configured from Settings at all; the assertions are unchanged and lifting the quarantine is a deliverable of LE-2124)
• Azure AI Foundry in the unified provider setup: configuration accepts deployment names, provider appears configured → core-functionality/model-provider/azure-ai-foundry-provider-setup.spec.ts (6 tests, all validated against a live Azure resource on 1.12.0.dev15 — 3/3 clean --retries=0 --workers=1 runs, every test force-failed. 4 of them need no Azure account and carry the surface on every lane: the two-variable form, the Foundry-only deployment hint asserted differentially against OpenRouter, the read-only unconfigured panel, an unresolvable endpoint rejected with valid:false and nothing persisted, and a deployment name absent from every catalog accepted and stored as Azure AI Foundry::llm::<name>. The other two — real credentials configuring the provider through Settings, and a real inference addressed by the deployment name — now run there too: the three AZURE_AI_FOUNDRY_* secrets are wired into the daily shard step and manual.yml's Docker job by #1270, proven by a dispatch that reported 6 passed where the lane had said 4 passed, 2 skipped. They still skip with the concrete reason wherever the credentials are absent — a local run without the .env block, pr-validation.yml and the external-URL job, both unwired on purpose: #1216 keeps an unrelated PR off that account's health, and #1055 keeps us from storing our credentials in an instance we do not own)

---

core-functionality/observability-monitoring/ — Tracing, Logs and Metrics

8.1 Traces

• View execution traces → core-functionality/observability-monitoring/traces.spec.ts
• Trace API returns paginated transactions → core-functionality/observability-monitoring/traces-detail.spec.ts
• Trace displays latency of each component → core-functionality/observability-monitoring/traces-latency-tokens.spec.ts (API totalLatencyMs, Flow Activity latency column, per-span latency in the Trace Details modal)
• Trace displays tokens consumed → core-functionality/observability-monitoring/traces-latency-tokens.spec.ts (API totalTokens + Flow Activity token column)
• Single-trace API returns 404 for an unknown trace_id → traces-detail-single.spec.ts
• Single-trace API returns the full TraceRead contract with a non-empty span tree → traces-detail-single.spec.ts
• Single-trace API returns populated tokenUsage + modelName on the LLM span (OpenAI) → traces-detail-llm-span-populated.spec.ts
• Bulk delete traces API returns 404 for an unknown flow_id → traces-delete.spec.ts
• Bulk delete traces API clears all traces for the flow (204 + empty list) → traces-delete.spec.ts
• Bulk delete of a trace with a populated span tree cascades (204, no FK violation) — regression #13955 → traces-delete-cascade.spec.ts
• Trace list filter ?status=error returns only the failing trace; ?status=<unknown> returns 422 → traces-list-filters.spec.ts
• Trace list filter ?status=ok returns only the successful trace → traces-list-filters.spec.ts
• Trace list filter ?start_time pins the >= lower bound (past hits, future misses) → traces-list-filters.spec.ts
• Trace list filter ?query=<substring> filters by trace name, incl. 50-char sanitize cap → traces-list-filters.spec.ts
• Trace list filter ?session_id filters by the session passed at run time → traces-list-filters.spec.ts

8.2 Notifications

• System notifications — build-success entry shows in the notifications tab → notifications.spec.ts
• Execution error notification → ui-ux/execution-error-notification.spec.ts
• Outdated component notification → core-components/outdated-component-notification.spec.ts

8.3 User State

• Track user progress → core-functionality/project-management/user-progress-track.spec.ts
• User flow state cleanup → flow-functionality/user-flow-state-cleanup.spec.ts

8.4 Error Handling and Edge Cases

• Component that raises Python error → core-components/validate-raise-errors-components.spec.ts
• Flow with error displays appropriate message → core-functionality/observability-monitoring/flow-error-message.spec.ts
• Network error during execution → ui-ux/execution-error-notification.spec.ts
• Execution timeout — clear message to user → ui-ux/execution-error-notification.spec.ts (transport-timeout path: route.abort("timedout") → "Workflow run failed" / "Failed to fetch"; the distinct deployed-flow "Run timed out. Please try again." is out of scope — see #694)

---

core-functionality/playground/ — Chat, Rendering and Output Tests

9.1 Chat Interactions

• Open Playground → exercised by every @stable playground spec via playground-btn-flow-io
• Send text message → exercised by playground-ux.spec.ts, playground-message-edit.spec.ts, playground-session-nav.spec.ts and others
• Receive LLM response → exercised by all specs that send a message via ChatInput → ChatOutput echo flow
• Response streaming (SSE) → core-functionality/playground/playground-response-streaming-sse.spec.ts
• Response polling → api/flows/api-build-polling-response.spec.ts
• Direct response → api/flows/api-build-direct-response.spec.ts
• Playground UX (playground-ux) → playground/playground-ux.spec.ts
• Send empty message — send button stays enabled by design (only disabled while a file upload is in progress) → playground/playground-empty-message-send.spec.ts
• [-] Send message while response is in progress — should wait or queue → playground/playground-send-while-in-progress.spec.ts
• Attach image in chat — compact preview appears in input before sending → core-functionality/playground/playground-output-image.spec.ts
• Image rendered in user message bubble after sending → core-functionality/playground/playground-output-image.spec.ts
• Attach non-image file (.txt) in chat — preview tile renders (delete button visible, no <img>) → core-functionality/playground/playground-non-image-attachment.spec.ts
• Non-image file rendered in user message after sending — truncated filename appears, no image emitted → core-functionality/playground/playground-non-image-attachment.spec.ts
• Attach multiple images — one compact preview per file is shown in the input → core-functionality/playground/playground-attachments-management.spec.ts
• Remove one of two attachments — the remaining preview stays intact → core-functionality/playground/playground-attachments-management.spec.ts
• Send with multiple attachments — all images render in the user message → core-functionality/playground/playground-attachments-management.spec.ts
• Remove the only attachment — input returns to empty state and text-only send still works → core-functionality/playground/playground-attachments-management.spec.ts
• Swap attachment (remove A, attach B) — only B is sent → core-functionality/playground/playground-attachments-management.spec.ts
• ChatInput Input Text pre-fills the playground textarea on first open → core-functionality/playground/playground-input-text-prefill.spec.ts
• ChatInput Input Text re-pre-fills the textarea on a new session → core-functionality/playground/playground-input-text-prefill.spec.ts
• Pre-filled Input Text can be sent as the first message of the session → core-functionality/playground/playground-input-text-prefill.spec.ts
• [-] Attach and send an image on a live LLM flow (Basic Prompting) — the image renders in the chat messages → core-functionality/llm-agents/chatInputOutputUser-shard-0.spec.ts
• [-] Custom sender_name on Chat Input/Output is applied to a live LLM turn — messages render as chat-message-<custom name> after a default-label turn → core-functionality/llm-agents/chatInputOutputUser-shard-2.spec.ts

9.2 History and Session

• Configure custom session ID → core-functionality/playground/playground-session-id.spec.ts
• Switch session — messages are isolated per session → core-functionality/playground/playground-session-nav.spec.ts
• Edit user message — hover reveals edit button, saved changes replace original text → core-functionality/playground/playground-message-edit.spec.ts
• Cancel message edit — original text is preserved → core-functionality/playground/playground-message-edit.spec.ts
• Message edited in playground is reflected in Session Logs → core-functionality/playground/playground-message-edit.spec.ts
• Clear chat removes all messages from Default Session (clear-chat-option via header menu) → core-functionality/playground/playground-session-clear.spec.ts
• Clear full session history (Default session) → playground/playground-clear-history.spec.ts
• Delete user-created session → playground/playground-clear-history.spec.ts
• History persists when reopening Playground → llm-agents/memory-history-regression.spec.ts, core-functionality/playground/playground-history-persist.spec.ts
• Rename unavailable for the Default Session → core-functionality/playground/playground-session-rename.spec.ts
• Rename unavailable for a session with no messages → core-functionality/playground/playground-session-rename.spec.ts
• Rename available and functional for a session with messages (Enter confirms, Escape cancels) → core-functionality/playground/playground-session-rename.spec.ts
• Create new session via new-chat button → core-functionality/playground/playground-session-nav.spec.ts
• Switch between sessions via session selector sidebar → core-functionality/playground/playground-session-nav.spec.ts
• Open Message Logs via session more-menu → core-functionality/playground/playground-message-logs.spec.ts
• Delete messages inside Message Logs table → core-functionality/playground/playground-message-logs.spec.ts
• Select individual session checkbox → reveals bulk-delete-button → core-functionality/playground/playground-bulk-delete.spec.ts
• Select all non-default sessions via select-all-checkbox → core-functionality/playground/playground-bulk-delete.spec.ts
• Bulk delete selected sessions → Default Session preserved → core-functionality/playground/playground-bulk-delete.spec.ts

9.3 Advanced Playground Features

• Playground fullscreen mode → playground/playground-fullscreen.spec.ts
• Shareable Playground — URL generation validated (switch enables sharing, href matches /playground/uuid) → playground/playground-shareable-url.spec.ts
• [!] Voice mode (voice assistant) → ui-ux/voice-assistant.spec.ts (all tests unconditionally skipped — spec is a stub)
• Stop button in Playground → core-functionality/playground/stop-button-playground.spec.ts

9.4 Output Modal

• Copy component output → core-functionality/playground/output-modal-copy-button.spec.ts
• Copy button in output → core-functionality/playground/output-modal-copy-button.spec.ts

9.5 Structured Data Output

• JSON Data output renders as code block → core-functionality/playground/playground-output-data.spec.ts
• DataFrame output renders as Markdown table → core-functionality/playground/playground-output-data.spec.ts

9.6 Human-in-the-Loop (1.11.0)

• Human Input suspends the run server-side, decision card renders in the Playground, Approve routes only the approved branch and the run leaves the suspended state; Reject routes only the reject branch → core-functionality/playground/human-input-pause-resume.spec.ts
• A suspended Human Input run is recoverable after a page reload (durable execution outliving the tab)

---

core-functionality/project-management/ — Project and Folder Management

10.1 Folder CRUD

• Create new folder → core-functionality/project-management/folder-crud.spec.ts
• Rename folder → core-functionality/project-management/folder-crud.spec.ts
• Delete empty folder → core-functionality/project-management/folder-crud.spec.ts
• Delete folder with flows inside → core-functionality/project-management/folder-crud.spec.ts
• Integrity after deletion — the deleted folder leaves the sidebar immediately, the page stays functional, and a sibling folder is untouched and still clickable → core-functionality/project-management/folder-deletion-integrity.spec.ts
• Create folder after deleting all folders — creating a folder right after a deletion works (no stale-cache collision) → core-functionality/project-management/folder-deletion-integrity.spec.ts
• [-] Deleting every folder lands on the empty-project screen (sidebar empty message + new_project_btn_empty_page) → core-functionality/project-management/folder-deletion-integrity.spec.ts (@destructive — account-wide wiper, runs only in the low-concurrency lane via PW_DESTRUCTIVE=1, see #1010; stays [-] permanently, since [x] requires @stable and @destructive must never carry it — the pair would mean "runs nowhere")
• [-] Upload flow by drag-and-drop to folder — dropping a collection file imports one flow per entry; dropping a single flow file imports exactly one → flow-functionality/dragAndDrop.spec.ts
• [-] Move flow to another folder

10.2 Folder Navigation

• [-] Navigate between folders → core-functionality/project-management/flow-navigation-between-folders.spec.ts
• [-] Search flow by name filters results correctly
• [-] Folders in navigation sidebar

---

core-functionality/templates/ — Predefined Flow and Component Models

11.1 Basic Templates

• [-] Basic Prompting (OpenAI)
• [-] Basic Prompting (Anthropic)
• [-] Simple Agent (OpenAI)
• [-] Simple Agent (Anthropic)
• [-] Simple Agent with memory
• [-] Vector Store RAG
• Memory Chatbot
• [-] Basic Prompting (OpenAI) → core/integrations/Basic Prompting.spec.ts
• [-] Basic Prompting (Anthropic) → core/integrations/Basic Prompting Anthropic.spec.ts
• [-] Simple Agent (OpenAI) → core/integrations/Simple Agent.spec.ts
• [-] Simple Agent (Anthropic) → core/integrations/Simple Agent Anthropic.spec.ts
• [-] Simple Agent with memory → core/integrations/Simple Agent Memory.spec.ts
• [-] Vector Store RAG → core/integrations/Vector Store.spec.ts
• Memory Chatbot → llm-agents/memory-history-regression.spec.ts

11.2 Content Generation Templates

• [-] Blog Writer
• [-] Instagram Copywriter
• [-] Twitter Thread Generator
• [-] SEO Keyword Generator
• [-] Portfolio Website Code Generator
• [-] SaaS Pricing

11.3 Analysis and Processing Templates

• [-] Document QA
• [-] Invoice Summarizer
• [-] Financial Report Parser
• [-] Image Sentiment Analysis
• [-] Text Sentiment Analysis
• [-] Youtube Analysis

11.4 Agent Templates

• [-] Dynamic Agent
• [-] Hierarchical Agent
• [-] Sequential Task Agent
• [-] Social Media Agent
• [-] Travel Planning Agent
• [-] Market Research
• [-] Research Translation Loop
• [-] Pokedex Agent
• [-] Price Deal Finder
• [-] News Aggregator

11.5 Advanced Templates

• [-] Custom Component Generator
• [-] Prompt Chaining
• [-] Decision Flow
• [-] Similarity
• [-] MCP Server (starter projects)

---

core-functionality/a2a/ — Agent-to-Agent Protocol (1.11.0)

⚠️ Every bullet below needs LANGFLOW_A2A_ENABLED=true on the instance — the flag is off by default (lfx/services/settings/groups/mcp.py), and with it off all three /api/v1/a2a/* routes answer 404 and the flow editor's Agent tab only renders "A2A is turned off on this server". Surface map, testability decisions and the full out-of-scope list: docs/core-functionality/a2a/a2a-coverage-scope.md (scoping issue #1195, upstream langflow-ai/langflow#13831, Jira epic LE-1588). Numbering starts at 16 because §12–§15 are already taken; the section sits here, with its area, on purpose.

16.1 A2A Server

• [-] Agent card served for a published flow — GET /api/v1/a2a/{flow_id}/.well-known/agent-card.json returns protocolVersion="0.3.0", url ending in /api/v1/a2a/{flow_id}/jsonrpc, capabilities.streaming=true, defaultInputModes=["application/json"], skills[0].id === flow_id, skills[0].tags=["langflow"] and an inputSchema object → core-functionality/a2a/a2a-server-agent-card.spec.ts
• [-] Card overrides applied — a2a_card_overrides (name / version / description / tags / examples) change exactly those card fields and nothing else → core-functionality/a2a/a2a-server-agent-card.spec.ts
• [-] Card gated on publication state — a2a_enabled=false, flow_type=workflow and an unknown flow id each return 404 (indistinguishable from an unmounted route, by design) → core-functionality/a2a/a2a-server-agent-card.spec.ts
• [-] Agent discovery list — GET /api/v1/a2a/agents lists the published flow with a cardUrl that resolves 200, omits a flow_type=workflow flow and an agent flow with a2a_enabled=false, and drops the row when the flow is unpublished (owner-scoped, not a cross-user directory) → core-functionality/a2a/a2a-server-discovery.spec.ts
• [-] JSON-RPC message/send round-trip — a per-run sentinel sent to a Chat Input→Chat Output passthrough comes back in the task's artifact text with state completed (no LLM involved) → core-functionality/a2a/a2a-server-jsonrpc-message-send.spec.ts
• [-] JSON-RPC error envelopes — an unknown method returns -32601 and a malformed envelope -32600/-32700, both over HTTP 200 (JSON-RPC-level errors are not HTTP errors here) → core-functionality/a2a/a2a-server-jsonrpc-message-send.spec.ts
• [-] Multi-turn context continuity — the first message/send response carries a server-minted contextId; reusing it on a second call returns the same contextId with a new task id and lands in the same stored session (session_id is the composite <uuid>:<contextId> in GET /api/v1/monitor/messages, carrying both turns as User/Machine pairs), while a call without it mints a different one → core-functionality/a2a/a2a-server-multi-turn-context.spec.ts
• [-] Task lifecycle — tasks/get reads the message/send task back with the same artifactId and status.timestamp (a read-back, not a re-run); an unknown id is -32001 "Task not found" and a cancel on a finished task -32002 "Task cannot be canceled" with the stored state untouched, both over HTTP 200; a task id cancelled through another flow's endpoint is -32001 (never -32002, which would confirm it exists); and a message/stream run cancelled mid-flight reports canceled from both tasks/cancel and tasks/get → core-functionality/a2a/a2a-server-tasks-lifecycle.spec.ts
• API-key auth gate on the JSON-RPC endpoint — with the flow's project set to auth_type=apikey, the card advertises the x-api-key scheme in securitySchemes, a call with no header returns 401 "API key required", a wrong key 401 "Invalid API key", and the owner's key 200 + completed (the gate LE-2081 lives behind; auth derives from the project, not the flow)
• [-] Agent tab publish flow — a blank flow shows Unavailable with the copy "Add a chat input and output to serve this flow." and cannot publish; adding Chat Input + Chat Output (unwired — the two node types are the gate) enables agent-publish-switch; publishing shows status Live and the agent-card-url that 404d as a draft now fetches 200; editing Name and adding a tag then pressing agent-save updates agent-card-name and changes name / skills[0].name / skills[0].tags on the card the API serves. The "Agent updated" toast is not asserted — measured transient (<3 s), so the durable pair is the status chip plus the served card → core-functionality/a2a/a2a-server-agent-tab-publish.spec.ts
• [~] Agent tab "Try it" panel — a sentinel sent from the panel over the live endpoint appears in agent-transcript twice (user turn + the agent's echo), the state reaches completed, the turn counter reads 1 turn and a Reset control is present → core-functionality/a2a/a2a-server-agent-tab-try-it.spec.ts. Partial: "View JSON-RPC exchange" is not covered because it is not implemented — agentTab.viewExchange appears exactly once per locale bundle in the shipped frontend (the dictionary entry) with zero call sites, and the string is absent from the DOM after a completed turn (measured on 1.12.0.dev14, #1244); pending an upstream question
• Non-owner cannot publish a flow — PATCH /api/v1/flows/{id} flipping a2a_enabled returns 403 "Cannot change a2a_enabled of a flow you do not own." (not automatable: needs two real users, and per-test user isolation is impossible under AUTO_LOGIN — measured in #1010)
• Disabled-server state — all three /api/v1/a2a/* routes 404 and the Agent tab renders "A2A is turned off on this server. Set LANGFLOW_A2A_ENABLED=true…" (out of scope by lane decision on #1195: the flag is on in every lane, and an off-lane cannot coexist with it in the same run; revisit only as a dedicated PW_A2A_OFF=1 lane)
• Push notification config and delivery — tasks/pushNotificationConfig/{set,get,list,delete} (out of reach: proving delivery needs a receiver with an inspectable inbox, which the self-hosted go-httpbin echo endpoint does not provide — LE-1706)
• JWS-signed agent cards (LE-1718) and rate limiting on the public endpoints (LE-1701) (out of reach: no URL-observable surface for the signature; driving the global v1 rate limits would make every parallel lane flaky)

16.2 A2A Client

• A2A Agent component, Internal mode — the mode=Internal dropdown lists the locally published agent and a run returns a Response containing the sentinel the published passthrough flow echoes (no LLM on either side)
• A2A Agent component, External mode — pointed at this instance's own card URL, the agent_card display renders the card chips (name; "Requires an API key" when the project is restricted) and a run returns the echoed sentinel; @regression for LE-1845 (NameError: name 'call_a2a_agent' is not defined). Blocked until a loopback self-call is allowed past Langflow's SSRF layer (LE-1904 class) — if it cannot be, LE-1845 stays uncovered
• A2A Agent used as a Tool — an Agent with the A2AAgent wired as a tool (tool_mode) calls the published agent and the reply reaches the playground; @regression for LE-1963 (self.user_id is None → "badly formed hexadecimal UUID string" on tool-approval resume). The only LLM-dependent bullet of the area — --workers=1 + models.json

---

flow-functionality/ — Graph Execution, Drag-and-Drop and JSON

12.1 Create Flow

• Create blank flow → flow-functionality/create-blank-flow.spec.ts
• Create flow from template → flow-functionality/create-flow-from-template.spec.ts
• Create flow by duplicating an existing one → flow-functionality/duplicate-flow.spec.ts
• Create flow via JSON file import → flow-functionality/export-import-flow.spec.ts

12.2 View and Edit Flow

• Rename flow via editor header → flow-functionality/flow-rename-header.spec.ts
• Rename flow and verify on main page listing → core-functionality/project-management/edit-flow-name.spec.ts
• Edit flow name and description → core-components/edit-name-description-node.spec.ts
• Flow auto-save on changes → flow-functionality/auto-save-off.spec.ts
• Flow settings → core-functionality/project-management/flowSettings.spec.ts

12.3 Delete Flow

• Delete individual flow → ui-ux/actionsMainPage-shard-1.spec.ts
• Delete multiple flows (bulk actions) → core-functionality/project-management/bulk-actions.spec.ts
• Shift-click range select + Ctrl/Cmd-click multi-select on main page → core-functionality/project-management/bulk-actions.spec.ts
• Bulk download selected flows → core-functionality/project-management/bulk-actions.spec.ts
• Confirm deleted flow does not appear in listing (after bulk delete) → core-functionality/project-management/bulk-actions.spec.ts

12.4 Export / Import Flow

• Export flow as JSON → flow-functionality/export-import-flow.spec.ts
• Exported JSON contains valid data.nodes structure → flow-functionality/export-import-flow.spec.ts
• Import flow via JSON file upload (drag-drop + upload button) → flow-functionality/export-import-flow.spec.ts
• [-] Import flow with outdated components → flow-functionality/import-outdated-flow.spec.ts
• Import invalid JSON — should display error message → flow-functionality/import-invalid-json.spec.ts

12.5 Flow Operations

• Lock flow — prevents editing → flow-functionality/lock-flow.spec.ts
• Unlock flow → flow-functionality/flow-lock.spec.ts
• [!] Move flow between folders via API — quarantined (#932): under concurrent writes PATCH /api/v1/flows/{id} answers 500 (sqlite3.OperationalError: database is locked on UPDATE flow SET folder_id) and the flow does not move; 14/24 at 2 concurrent clients, 0/30 serial. Same root cause as #965, not a separate one — the daily artifact shows the failing assert is expect(patchRes.status()).toBe(200) receiving 500, not a stale folder_id. Product defect tracked under LE-2020; the 200 assertion is unchanged → api/flows/api-folders-crud.spec.ts
• Publish flow → flow-functionality/publish-flow.spec.ts
• Save flow components as template → core-components/saveComponents.spec.ts

12.6 Flow Execution

• [!] Run Flow component executes another flow — runs again after the #966 quarantine lift, but not @stable while the upstream New Flow dead-click defect (LE-2019) is open; the shared helper gates on the flows list having rendered so the suite stays out of the broken window → flow-functionality/run-flow.spec.ts
• Run a flow from the canvas — terminal-node run builds the whole graph; all nodes reach build success and output is produced → flow-functionality/flow-execution-canvas.spec.ts
• Stop building flow → flow-functionality/stop-building.spec.ts
• [!] Playground button disabled with empty flow — needs review → regression/flow-functionality/generalBugs-shard-3.spec.ts (test skipped: assertion was a no-op, current Langflow behavior to confirm)

---

mcp/ — Model Context Protocol

⚠️ Tests that execute agents via MCP must use SimpleAgentTemplatePage and models.json. See CLAUDE.md in this folder for the complete guide.

mcp/client/ — Tool and Context Consumption

13.1 MCP Client

• Configure connection with external MCP server (stdio or HTTP) → mcp/client/mcp-client-regression.spec.ts
• List available tools via MCP protocol → mcp/client/mcp-client-regression.spec.ts
• Execute MCP server tool and receive result in flow → mcp/client/mcp-client-regression.spec.ts
• MCP server connection error — unreachable server produces empty tool dropdown → mcp/client/mcp-client-regression.spec.ts
• Configure connection via HTTP form tab → mcp/client/mcp-client-regression.spec.ts
• [-] configureMcpServer helper registers an MCP server via the HTTP form → core-components/configure-mcp-and-custom-component.spec.ts
• Execute numeric tool with inputs and verify result → mcp/client/mcp-client-regression.spec.ts
• Duplicate MCP server registration returns 409 Conflict → mcp/client/mcp-server-registration-status-codes.spec.ts
• Deleting a non-existent MCP server returns 404 Not Found → mcp/client/mcp-server-registration-status-codes.spec.ts
• Agent uses MCPTools as tool and calls echo via MCP → mcp/client/mcp-client-agent.spec.ts
• Gemini × MCP tool-calling regression — agent invokes the echo MCP tool (regression for fixed upstream #440) → mcp/client/mcp-client-agent-gemini-tool-regression.spec.ts
• List available resources via MCP protocol (client not-implementable on 1.11.x — MCPTools component and v2 client API expose tools only; server-side resources covered in §14.1 → mcp/server/mcp-server-resources.spec.ts)
• Consume resource URI and inject content into flow (client not-implementable on 1.11.x — no client resource surface; server-side read covered in §14.1 → mcp/server/mcp-server-resources.spec.ts)

---

mcp/server/ — Resource and Tool Provider

14.1 MCP Server

• MCP Server tab in flow → mcp/server/mcp-server-tab.spec.ts
• Add MCP server via modal → mcp/server/mcp-server-tab.spec.ts
• Starter project with MCP → mcp/server/mcp-server-starter-projects.spec.ts
• Flow exposed as MCP server — verify generated endpoint → mcp/server/mcp-server-protocol.spec.ts
• Execute MCP server tool via MCP protocol → mcp/server/mcp-server-protocol.spec.ts
• Register an external MCP server through the stdio form — command + args resolves the server's real tools into the MCPTools node → mcp/server/mcp-server.spec.ts
• Add-server modal fields persist across save → reopen-for-edit — stdio (name, command, 4 args, 2 env pairs) and HTTP/SSE (name, URL, 2 headers, 2 env pairs) → mcp/server/mcp-server.spec.ts
• Tool list refreshes when a registered server is edited to run a different package → mcp/server/mcp-server.spec.ts
• stdio command must be a single executable — a command with an embedded argument is refused and the same registration split into command + args is accepted (upstream hardening #14073; #1091) → mcp/server/mcp-server.spec.ts
• The project's own Streamable HTTP endpoint registers as an MCP server and exposes its flows as tools → mcp/server/mcp-server.spec.ts
• [-] Resource exposed by server is accessible via URI — flow files are exposed as MCP resources: resources/list is @stable; resources/read blocked by a live Langflow regression on 1.12.x (AttributeError: 'str' object has no attribute 'hex', filed upstream LE-2012) — kept as a guard, not promoted → mcp/server/mcp-server-resources.spec.ts
• Prompt exposed by server returns correct template (no product surface on 1.11.x — MCP server prompts/list returns []; #829)

---

ui-ux/ — Visual Interface, Canvas and Design System

15.1 Component Sidebar

• Search component by name — matching component listed AND non-matching one hidden, case-insensitive → ui-ux/sidebar-search-and-filter.spec.ts
• [~] Hover over component shows tooltip/preview — no product surface on 1.12.0.dev6: hovering a sidebar card renders zero [role="tooltip"] elements, zero Radix poppers, no title and no aria-describedby; the only hover affordance is the + button, already covered by core-components/componentHoverAdd.spec.ts (#937)
• Keyboard search (keyboard shortcut) — / focuses the search, Tab reaches a result, Space/Enter add that exact component → ui-ux/keyboardComponentSearch.spec.ts
• Filter components by category — disclosure collapse/expand and non-matching categories removed while filtering → ui-ux/sidebar-search-and-filter.spec.ts
• [~] Sidebar shows correct provider count — no count surface on 1.12.0.dev6: the sidebar renders no numeric badge or count text anywhere (the count lives on Settings → Model Providers, §7). What exists is grouping under disclosure-bundles-<provider>, covered by ui-ux/sidebar-search-and-filter.spec.ts (#937)

15.2 Add Components to Canvas

• Drag component from sidebar to canvas — node lands at the drop position → ui-ux/sidebar-add-component.spec.ts
• Double-click in sidebar adds component to canvas → ui-ux/sidebar-add-component.spec.ts
• Hover + click "+" button adds component to canvas → core-components/componentHoverAdd.spec.ts
• Added component appears with default settings — every field value compared against the GET /api/v1/all catalog template → ui-ux/sidebar-add-component.spec.ts

15.3 Component Connections

• Connect two compatible components — clicking the Chat Input source handle then the Chat Output target handle creates exactly one edge, persisted to data.edges; repeating the pair does not duplicate it → flow-functionality/canvas-connect-components.spec.ts
• Prevent connection between incompatible types — a DataFrame output (Split Text chunks) into a Message input (Chat Output) creates no edge, while the Message→Message pair in the same test does (positive control); target-to-target is separately covered as invalid topology → flow-functionality/canvas-connect-components.spec.ts
• Delete edge/connection — right-clicking the edge context menu and choosing the destructive item removes the edge from the canvas and from data.edges → flow-functionality/canvas-edge-reconnect.spec.ts
• Filter edges by data type — clicking an input handle filters the sidebar to compatible sources (url → string sources, headers → Data sources), and the legacy/beta toggles expand that set → ui-ux/filterSidebar.spec.ts
• Reconnect existing edge — after deleting it, clicking the same two handles restores exactly one edge in the canvas and in the flow → flow-functionality/canvas-edge-reconnect.spec.ts

15.4 Node Manipulation

• Delete component from canvas via Backspace key → core-components/componentDelete.spec.ts
• Delete component from canvas via node options (...) menu → core-components/componentDelete.spec.ts
• Copy and paste ChatOutput component (Ctrl+C / Ctrl+V) → flow-functionality/canvas-copy-paste.spec.ts
• Copy and paste Prompt Template (component with dynamic ports) (Ctrl+C / Ctrl+V) → flow-functionality/canvas-copy-paste.spec.ts
• Canvas keyboard shortcuts — Duplicate/Delete/Copy/Paste/Cut/Undo/Redo each act on the selected node, with the selection re-gated before every keypress → ui-ux/langflowShortcuts.spec.ts
• Minimize component on canvas — the options menu collapses the node (every handle gains no-show, height shrinks) and persists data.showNode = false; the item swaps to Expand, which restores both; four further minimize/expand cycles keep both states correct and the persisted showNode in step (#1290) → ui-ux/minimize.spec.ts
• Move component within canvas — dragging a node by its title moves it on canvas and the new coordinates reach the backend (GET /api/v1/flows/{id} position matches the rendered transform) → flow-functionality/canvas-move-node.spec.ts
• Select multiple components via box selection — a Shift+drag marquee enclosing two separated nodes takes .react-flow__node.selected from 0 to 2, while a marquee drawn away from them selects nothing (negative control) → flow-functionality/canvas-multiselect.spec.ts
• Delete multiple selected components (marquee box selection) → core-components/componentDelete.spec.ts, flow-functionality/canvas-multiselect.spec.ts
• Deselect node by clicking on empty canvas area — clicking .react-flow__pane clears a selection asserted present first → flow-functionality/canvas-deselect-node.spec.ts
• Deselect node via Escape → flow-functionality/canvas-deselect-node.spec.ts

15.5 Canvas Zoom and Navigation

• Zoom in / Zoom out → ui-ux/canvas-zoom-navigation.spec.ts
• Fit View centers nodes → ui-ux/canvas-zoom-navigation.spec.ts
• Fit View button in toolbar → ui-ux/canvas-zoom-navigation.spec.ts
• Scroll to navigate canvas — on 1.12 the wheel zooms anchored at the pointer (no scroll-to-pan surface) → ui-ux/canvas-zoom-navigation.spec.ts
• [~] Minimap — feature flag-gated

15.6 Grouping

• Create component group → core-components/nested-grouping-regression.spec.ts
• Ungroup components → core-components/nested-grouping-regression.spec.ts
• Expand/collapse group → core-components/nested-grouping-regression.spec.ts

15.7 Freeze and State

• Freeze component — a frozen component serves its cached output instead of recomputing → flow-functionality/freeze-and-state.spec.ts
• Freeze path — freezing a component also freezes every component upstream of it → flow-functionality/freeze-and-state.spec.ts
• Unfreeze component — releases the whole path and the component recomputes → flow-functionality/freeze-and-state.spec.ts

15.8 Sticky Notes

• Add sticky note — the canvas-add-note-button canvas control places a note at the default 280×140 and the flow gains a node with type: "noteNode" → ui-ux/sticky-notes.spec.ts
• Edit sticky note text → ui-ux/edit-sticky-note-text.spec.ts
• Change sticky note color — the picker offers all seven presets; choosing rose repaints the note (--note-rose inline style) and persists template.backgroundColor: "rose" → ui-ux/sticky-notes.spec.ts
• Resize sticky note — dragging the bottom-right resize handle grows the note past 280×140 and the new dimensions persist to the node's width/height → ui-ux/sticky-notes.spec.ts
• Delete sticky note — removed via the options menu and via Backspace, gone from the canvas and from the persisted flow; deleting one of two leaves exactly one → flow-functionality/canvas-sticky-note-delete.spec.ts

15.9 Right-Click and Menus

• [~] Context menu via right-click on canvas — no product surface on 1.12.0.dev8: a right-click on .react-flow__pane dispatches a contextmenu event that reaches document with defaultPrevented === false (so the browser's native menu is what opens) and renders zero [role="menu"] / [role="listbox"] / Radix popper elements, with or without a node selected — Langflow does not wire ReactFlow's onPaneContextMenu. That absence is now guarded negatively (the pane right-click dismisses an open node menu, opens nothing, and leaves the selection intact) → ui-ux/right-click-dropdown.spec.ts. Edge menus (edge-context-menu-trigger) are a left-click affordance and belong to §15.3 (#945, #1027)
• Context menu via right-click on component — one right-click selects the node AND opens its options menu with the exact ordered item contract (Save/Duplicate/Copy/Docs/Minimize/Freeze/Download/Delete), Escape closes it, and choosing Duplicate from that menu adds the node → ui-ux/right-click-dropdown.spec.ts
• Main menu actions — the header account menu lists all nine menu_*_button items, its version row matches GET /api/v1/version, the four external items carry their documented href/target=_blank, Escape closes it, and the Settings item routes to /settings → ui-ux/main-menu-actions.spec.ts

15.10 Settings and UI Configuration

• Access Settings page — profile menu opens Settings, the sidebar lists General/Model Providers/Shortcuts/Messages, and each section renders its own content (General shows the Language + Profile Picture groups) → ui-ux/settings-navigation.spec.ts, ui-ux/settings-general-section.spec.ts
• Message history settings — Settings → Messages grid keeps the 11-column contract, renders messages oldest-first (1.12 get_messages defaults to order=ASC) and the sender "Equals User" filter narrows/restores the row set → ui-ux/settings-message-history.spec.ts
• Change appearance/theme settings — dark/light toggle updates #body.dark class → ui-ux/settings-theme-toggle.spec.ts
• Keyboard shortcuts work in editor — Duplicate/Delete/Copy/Paste/Cut/Undo/Redo each act on the selected node (node count asserted after every keypress) → ui-ux/langflowShortcuts.spec.ts
• [~] All documented shortcuts work — all 27 defaultShortcuts rows are listed with a non-empty key binding in Settings → Shortcuts (ui-ux/settings-navigation.spec.ts), and 7 of them are exercised on canvas (ui-ux/langflowShortcuts.spec.ts) plus 1 rebound end-to-end (ui-ux/settings-shortcuts-edit.spec.ts); the remaining 20 (API, Docs, Download, Play, Group, Minimize, Freeze, Save, Code, Update, Controls, sidebar search, …) are not exercised yet
• Edit a keyboard shortcut (Duplicate → Ctrl/Cmd+Alt+U) persists to the table and the new combination triggers the action on canvas → ui-ux/settings-shortcuts-edit.spec.ts
• API Keys table renders created_at/expires_at in the viewer's local timezone (UTC→local), shows "Never" for unused keys and ∞ for no-expiry keys (PR #13471) → ui-ux/api-keys-timezone-display.spec.ts

---

Coverage Summary — Test Automation Coverage

Validated = test carries the @stable tag. Needs validation = automated but not yet @stable (bug, flake under investigation, or pending team review).

Module	Total	Validated [x]	Needs validation [-]	Partial [~]/[!]	Not automated [ ]
api/flows/ — REST API	28	27	0	1	0
core-components/ — Component Config	27	24	3	0	0
core-components/ — Core Components	91	87	3	0	1
core-functionality/auth/	21	7	13	1	0
core-functionality/knowledge-ingestion/	8	8	0	0	0
core-functionality/llm-agents/	40	32	5	1	2
core-functionality/model-provider/	34	31	2	1	0
core-functionality/observability-monitoring/	24	24	0	0	0
core-functionality/playground/	52	47	3	1	1
core-functionality/project-management/	12	6	6	0	0
core-functionality/templates/	41	2	39	0	0
core-functionality/a2a/	18	0	9	1	8
flow-functionality/	28	24	1	3	0
mcp/client/	13	10	1	0	2
mcp/server/	12	10	1	0	1
ui-ux/ — Canvas	44	40	0	4	0
ui-ux/ — Settings	7	6	0	1	0
TOTAL	500	385 (77%)	86 (17%)	14 (3%)	15 (3%)

Note: Validated [x] counts checklist bullets, not test() calls. The @stable tag is per-test(), and a single @stable test may map to several bullets via test.step() (e.g. the agent suite covers 7 bullets). The canonical list of @stable test() calls is in Phase 0 — Validated below.

---

Implementation Roadmap

---

🟢 Phase 0 — Validated

447 test() calls carrying the @stable tag, distributed across 176 spec files. Run weekly by the stable workflow. New specs are merged with all tests tagged @stable; the tag is removed per-test during weekly triage when a failure is classified as a test bug — so a spec may end up with a mix of tagged and untagged tests over time.

api/flows/

• direct event_delivery streams build events inline (no job_id) and echoes the input → api-build-direct-response.spec.ts
• direct is distinct from the job_id path: streaming delivery returns a job_id → api-build-direct-response.spec.ts
• polling is the two-step path: POST returns a job_id shell (no inline events) → api-build-polling-response.spec.ts
• the poll loop drains the build to completion across repeated GET /events calls → api-build-polling-response.spec.ts
• API Request component — include_httpx_metadata=true adds request headers to output → api-component-regression.spec.ts
• API Request component — timeout error returns status_code 500 with error field → api-component-regression.spec.ts
• POST /api/v1/custom_component returns valid component structure → api-custom-component-creation.spec.ts
• POST /api/v1/custom_component with invalid code returns error → api-custom-component-creation.spec.ts
• GET /api/v1/all includes component types → api-custom-component-creation.spec.ts
• POST /api/v1/custom_component without auth returns 401 or 403 → api-custom-component-creation.spec.ts
• POST creates flow and returns ID → api-flows-crud.spec.ts
• GET lists flows and includes the created one → api-flows-crud.spec.ts
• GET by ID returns correct flow → api-flows-crud.spec.ts
• PATCH updates flow name and description → api-flows-crud.spec.ts
• DELETE removes flow and returns 200 → api-flows-crud.spec.ts
• GET after DELETE returns 404 → api-flows-crud.spec.ts
• GET non-existent flow returns 404 → api-flows-crud.spec.ts
• POST with missing name returns 422 → api-flows-crud.spec.ts
• deleted flow does not appear in flows listing → api-flows-crud.spec.ts
• POST creates folder and returns ID and name → api-folders-crud.spec.ts
• GET lists folders and includes the created one → api-folders-crud.spec.ts
• GET /health_check returns 200 with status ok → api-health-check.spec.ts
• GET /health_check returns db ok → api-health-check.spec.ts
• GET /health_check responds within 5 seconds → api-health-check.spec.ts
• GET /health_check response has correct content-type → api-health-check.spec.ts
• POST /api/v1/flows/ with invalid Bearer token returns 401, 403, or 422 → api-invalid-key.spec.ts
• GET /api/v1/flows/ without Authorization header returns 401 or 403 → api-invalid-key.spec.ts
• GET /api/v1/flows/{id} with invalid Bearer token returns 401 or 403 → api-invalid-key.spec.ts
• POST /api/v1/run/{id} with invalid x-api-key returns 401 or 403 → api-invalid-key.spec.ts
• DELETE /api/v1/flows/{id} without Authorization header returns 401 or 403 → api-invalid-key.spec.ts
• PATCH /api/v1/flows/{id} with wrong token does not update the flow → api-invalid-key.spec.ts
• rejects an expired API key with 403 and accepts a valid one with 200 → api-key-expiry-enforcement.spec.ts
• evaluates the expiry boundary in UTC, not shifted by the viewer offset → api-key-expiry-enforcement.spec.ts
• returns 200 with array → api-monitor-messages.spec.ts
• without auth returns 401 or 403 → api-monitor-messages.spec.ts
• filtered by session_id returns only matching messages → api-monitor-messages.spec.ts
• filtered by flow_id returns only matching messages → api-monitor-messages.spec.ts
• combined session_id and flow_id filters return 200 → api-monitor-messages.spec.ts
• messages contain required fields when not empty → api-monitor-messages.spec.ts
• executes flow with input_value and returns outputs → api-run-flow.spec.ts
• executes flow with custom session_id and persists messages under it → api-run-flow.spec.ts
• returns 404 for non-existent flow ID → api-run-flow.spec.ts
• tweaks override a component field at runtime → api-run-with-tweaks.spec.ts
• empty tweaks object is a no-op and leaves the flow default in effect → api-run-with-tweaks.spec.ts
• tweaks referencing a non-existent component are silently ignored → api-run-with-tweaks.spec.ts
• GET /api/v1/version returns 200 with a non-empty version string → api-version.spec.ts
• GET /api/v1/version reports the Langflow package and main_version → api-version.spec.ts
• GET /api/v1/version response has correct content-type → api-version.spec.ts
• GET /api/v1/version responds within 5 seconds → api-version.spec.ts
• POST /api/v1/version returns 405 Method Not Allowed → api-version.spec.ts

core-components/

• renders on canvas with default fields and handles → agent-component-regression.spec.ts
• system prompt accepts input and persists across flow reload → agent-component-regression.spec.ts
• model dropdown exposes manage-model-providers and lists configured models → agent-component-regression.spec.ts
• selecting a different-provider model swaps the canvas provider icon → agent-component-regression.spec.ts
• API Request component — renders on canvas with correct output and URL handles → api-request-component-regression.spec.ts
• API Request component — inspector fields accept configured values → api-request-component-regression.spec.ts
• API Request component — invalid URL is accepted by field and run shows error notification → api-request-component-regression.spec.ts
• API Request component — GET request returns 200 and output Data contains all required fields → api-request-component-regression.spec.ts
• API Request component — POST method executes POST verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — PUT method executes PUT verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — PATCH method executes PATCH verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — DELETE method executes DELETE verb and returns 200 → api-request-component-regression.spec.ts
• API Request component — non-2xx HTTP response propagates status_code without crashing → api-request-component-regression.spec.ts
• API Request component — query parameters embedded in URL are sent and echoed → api-request-component-regression.spec.ts
• API Request component — inspector headers table accepts key + value cell entries → api-request-component-regression.spec.ts
• API Request component — cURL tab switches mode and field accepts a cURL command → api-request-component-regression.spec.ts
• API Request component — cURL mode parses command, auto-fills URL, executes GET and returns 200 → api-request-component-regression.spec.ts
• API Request component — body table accepts key + value cell entries when method is POST → api-request-component-regression.spec.ts
• API Request component — flow state persists in database after autosave (URL, method, headers) → api-request-component-regression.spec.ts
• Show Beta Components toggle controls visibility of beta components in the sidebar → beta-components-toggle-regression.spec.ts
• Chat Input — toggling showfiles exposes the Files inspector field → chat-input-files-field-regression.spec.ts
• Chat Input — uploading via the inspector populates the Files field → chat-input-files-field-regression.spec.ts
• Chat Input → Chat Output — inspector-attached file is rendered in the Playground message → chat-input-files-field-regression.spec.ts
• Chat Input — clicking the dismiss button on the Files field clears the value → chat-input-files-field-regression.spec.ts
• Chat Input component — renders on canvas with Message output handle and Input Text field → chat-input-output-component-regression.spec.ts
• Chat Output component — renders on canvas with Inputs handle and run button → chat-input-output-component-regression.spec.ts
• Chat Input → Chat Output connection is accepted on canvas (Message ↔ Message) → chat-input-output-component-regression.spec.ts
• Chat Input → Chat Output — Input Text value propagates to ChatOutput on run → chat-input-output-component-regression.spec.ts
• Chat Input — sender_name override is reflected in the Playground chat message → chat-input-output-component-regression.spec.ts
• Chat Input/Output — default sender_name is 'User' on input and 'AI' on output → chat-input-output-component-regression.spec.ts
• breaking-change outdated components alert with a Review action, not a silent Update → component-breaking-change-alert.spec.ts
• reviewing a single breaking change warns about disconnection and defaults to a backup → component-breaking-change-alert.spec.ts
• Review All flags every outdated component as breaking and pre-selects none → component-breaking-change-alert.spec.ts
• Should delete a single component with the Backspace key → componentDelete.spec.ts
• Should delete a single component via the node options menu → componentDelete.spec.ts
• Should delete multiple selected components with a marquee selection → componentDelete.spec.ts
• user can add components by hovering and clicking the plus icon → componentHoverAdd.spec.ts
• custom component code button should be pink when adding custom component → customComponentAdd.spec.ts
• Data Operations Text mode returns the Case Conversion result as a Message → data-operations-component.spec.ts
• Data Operations Word Count switches the Text-mode output to JSON and counts the text → data-operations-component.spec.ts
• Data Operations JSON mode selects a single key from an upstream JSON output → data-operations-component.spec.ts
• Data Operations Table mode filters the rows of an upstream Table output → data-operations-component.spec.ts
• All three legacy operations components name Data Operations as their replacement → data-operations-legacy-link.spec.ts
• The legacy banner link filters the sidebar to Data Operations → data-operations-legacy-link.spec.ts
• Searching a legacy operations name surfaces Data Operations with legacy components hidden → data-operations-legacy-link.spec.ts
• A legacy operations component still builds and returns its result → data-operations-legacy-link.spec.ts
• two API Request nodes expose the same field without duplicating its DOM id → duplicate-dom-ids-regression.spec.ts
• two Agent nodes expose the same field without duplicating its DOM id → duplicate-dom-ids-regression.spec.ts
• user can edit a URL tool action in Tool Mode and the edits persist → edit-tools.spec.ts
• a full custom component built from code exposes its declared interface → full-custom-component.spec.ts
• the system must delete the handles from advanced fields when the code is updated → general-bugs-delete-handle-advanced-input.spec.ts
• any changes on the node must be saved on user interaction → general-bugs-save-changes-on-node.spec.ts
• Human Input renders the default Approve and Reject branch handles when added to the canvas → human-input-node-config.spec.ts
• adding a custom User Action creates its branch handle without a reload → human-input-node-config.spec.ts
• the configured branch handles persist after save and reload → human-input-node-config.spec.ts
• If-Else routes matching input through the True branch and skips the False branch → if-else-component-regression.spec.ts
• If-Else routes non-matching input through the False branch and skips the True branch → if-else-component-regression.spec.ts
• If-Else operator=contains routes a substring match through the True branch → if-else-component-regression.spec.ts
• If-Else operator=regex routes a valid pattern match through the True branch → if-else-component-regression.spec.ts
• If-Else operator=regex hides the case_sensitive advanced field → if-else-component-regression.spec.ts
• If-Else case_sensitive defaults to ON — mixed-case inputs route to the False branch → if-else-component-regression.spec.ts
• If-Else with case_sensitive=OFF treats mixed-case inputs as a match (True branch) → if-else-component-regression.spec.ts
• If-Else operator=greater than routes a numeric match (10 > 5) through the True branch → if-else-component-regression.spec.ts
• If-Else operator=less than routes a numeric match (2.5 < 10) through the True branch → if-else-component-regression.spec.ts
• If-Else operator=less than or equal routes an equal-operands match (5 <= 5) through the True branch → if-else-component-regression.spec.ts
• If-Else operator=greater than or equal routes an equal-operands match (5 >= 5) through the True branch → if-else-component-regression.spec.ts
• Show Legacy Components toggle controls visibility of legacy components in the sidebar → legacy-components-toggle-regression.spec.ts
• Loop component — renders correctly with all handles and output inspection buttons → loop-component-regression.spec.ts
• Loop component — run without connections shows build failed notification → loop-component-regression.spec.ts
• Loop component — stops after exhausting input DataFrame and emits aggregated done → loop-component-regression.spec.ts
• box-selecting two connected non-IO components and clicking Group collapses them into a single Group node → nested-grouping-regression.spec.ts
• ungrouping a Group node restores the original components and the edge between them → nested-grouping-regression.spec.ts
• importing a flow with outdated components raises the flow-level outdated notification → outdated-component-notification.spec.ts
• the outdated-notification count matches the per-node update indicators → outdated-component-notification.spec.ts
• text input field edit persists → parameters-panel-field-types.spec.ts
• dropdown field edit persists → parameters-panel-field-types.spec.ts
• textarea field edit persists → parameters-panel-field-types.spec.ts
• int field edit persists → parameters-panel-field-types.spec.ts
• tab field edit persists → parameters-panel-field-types.spec.ts
• toggle field edit persists → parameters-panel-field-types.spec.ts
• float field edit persists → parameters-panel-field-types.spec.ts
• slider field edit persists → parameters-panel-field-types.spec.ts
• code field edit persists → parameters-panel-field-types.spec.ts
• table field edit persists → parameters-panel-field-types.spec.ts
• key-pair field edit persists → parameters-panel-field-types.spec.ts
• input list field edit persists → parameters-panel-field-types.spec.ts
• Prompt Template component — renders on canvas with output handle → prompt-template-component-regression.spec.ts
• Prompt Template component — variables in curly braces generate dynamic input handles → prompt-template-component-regression.spec.ts
• Prompt Template component — removing a variable removes its input handle → prompt-template-component-regression.spec.ts
• Prompt Template component — replacing a variable updates handles accordingly → prompt-template-component-regression.spec.ts
• Prompt Template component — clearing the template removes all dynamic handles → prompt-template-component-regression.spec.ts
• Prompt Template component — modal edits persist in UI and in saved flow → prompt-template-component-regression.spec.ts
• Prompt Template — use_double_brackets toggle is exposed in the InspectionPanel with its upstream display name → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — default toggle state is OFF; f-string mode extracts {var} and treats {{var}} as literal → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — enabling toggle switches parser to mustache mode; {{var}} creates handle and {var} is ignored → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — disabling toggle reverts to f-string mode and variables are re-extracted under the new parser → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — use_double_brackets value persists in the autosaved flow → prompt-template-double-brackets-regression.spec.ts
• Prompt Template — mustache {{ var }} (spaces inside braces) is rejected with an error toast and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — mustache {{var.attr}} (dot notation) is rejected with an error toast and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — mustache {{#section}}{{/section}} is rejected with the complex-syntax message and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — mustache {{{var}}} (triple braces) is rejected with the complex-syntax message and creates no handle → prompt-template-invalid-mustache-patterns-regression.spec.ts
• Prompt Template — {var.attr} (dot notation) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {var name} (space inside identifier) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {var,name} (comma inside identifier) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {1var} (leading digit) is rejected with an error toast and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — {} (empty braces) is accepted by the parser and creates no handle → prompt-template-invalid-patterns-regression.spec.ts
• Prompt Template — repeating the same variable produces exactly one handle (deduplication contract) → prompt-template-invalid-patterns-regression.spec.ts
• saving a canvas component as a template makes it reusable from the sidebar → saveComponents.spec.ts
• should allow only one Chat Input on the canvas → singleton-components.spec.ts
• should not allow adding a Webhook while a Chat Input is on the canvas → singleton-components.spec.ts
• should not allow duplicating a Chat Input → singleton-components.spec.ts
• should not allow copying and pasting a Chat Input → singleton-components.spec.ts
• should allow only one Webhook on the canvas → singleton-components.spec.ts
• should not allow adding a Chat Input while a Webhook is on the canvas → singleton-components.spec.ts
• should not allow duplicating a Webhook → singleton-components.spec.ts
• should not allow copying and pasting a Webhook → singleton-components.spec.ts
• a component in Tool Mode can be grouped with its Agent consumer → tool-mode-group.spec.ts
• User should be able to use components as tool → tool-mode.spec.ts
• applying a single component update refreshes it, decrements the outdated count, and creates a backup → update-component-action.spec.ts
• user should be able to see errors on popups when raise an error → validate-raise-errors-components.spec.ts
• Webhook component — HTTP POST accepts JSON and plain-text bodies returning 202 → webhook-component-regression.spec.ts
• Webhook component — flow is saved to database and contains the Webhook node → webhook-component-regression.spec.ts
• Webhook component — cURL command in inspector shows valid POST URL with flow ID → webhook-component-regression.spec.ts
• Webhook component — empty data field returns empty Data object → webhook-component-regression.spec.ts
• Webhook component — endpoint field renders the actual webhook URL → webhook-component-regression.spec.ts
• Webhook component — copy button copies the endpoint URL to clipboard → webhook-component-regression.spec.ts
• Webhook component — POST to non-existent flow name returns 404 → webhook-component-regression.spec.ts
• Webhook component — valid JSON payload is propagated as structured Data output → webhook-component-regression.spec.ts
• Webhook component — invalid JSON payload is encapsulated in {payload: ...} → webhook-component-regression.spec.ts
• GET /api/v1/monitor/messages returns 200 with array response → webhook-component-regression.spec.ts

core-functionality/auth/

• logout must redirect user to login page → logout-flow.spec.ts
• after logout, navigating to root must redirect to login → logout-flow.spec.ts
• after logout, reload must stay on login page → logout-flow.spec.ts

core-functionality/knowledge-ingestion-management/

• upload a file through the Files page → file-types-upload.spec.ts
• should navigate to Files page and expose upload affordances → files-page.spec.ts
• should upload file using upload button → files-page.spec.ts
• should upload file using drag and drop → files-page.spec.ts
• should upload multiple files with different types → files-page.spec.ts
• should search uploaded files → files-page.spec.ts
• should handle bulk actions for multiple files → files-page.spec.ts
• user should not be able to upload a file larger than the limit → limit-file-size-upload.spec.ts
• Full RAG pipeline grounds the model answer on the retrieved chunk → rag-pipeline.spec.ts
• Split Text splits an ingested document into the expected number of chunks → split-text-chunking.spec.ts
• upload a file through the Read File component and read its content → upload-via-component.spec.ts
• Knowledge Base indexes the ingested document chunks (available for query) → vector-store-index-query.spec.ts
• Knowledge Base query returns the relevant chunk for the prompt → vector-store-index-query.spec.ts

core-functionality/llm-agents/

• agent interaction suite → agent-component-regression.spec.ts
• agent stop button must halt execution mid-run → agent-component-regression.spec.ts
• Agent settings survive save and reopen → agent-config-persistence.spec.ts
• context-scoped retrieval returns all turns of the context and not the untagged control → agent-context-id-continuity.spec.ts
• agent run persists every session message tagged with the custom context_id → agent-context-id-continuity.spec.ts
• mirrored context-scoped retrievals return only their own context's messages → agent-context-id-isolation.spec.ts
• switching the agent's context_id re-tags new turns without touching previous ones → agent-context-id-isolation.spec.ts
• toggle ON (default): agent's date tool returns today's date → agent-current-date-tool.spec.ts
• toggle OFF: the date tool is removed from the agent's toolkit → agent-current-date-tool.spec.ts
• model refusal does not crash the component → agent-empty-refusal-response.spec.ts
• empty response does not crash the component → agent-empty-refusal-response.spec.ts
• input via ChatInput handle drives the agent response → agent-input-sources.spec.ts
• input via the Agent's direct field drives the agent response → agent-input-sources.spec.ts
• causal control — a high max iterations does not hit the limit → agent-max-iterations.spec.ts
• max_tokens=50 caps the response's output tokens → agent-max-tokens.spec.ts
• causal control — unset max_tokens generates freely → agent-max-tokens.spec.ts
• selecting 'Connect other models' clears the previously selected model → agent-model-connection-isolation.spec.ts
• agent selects the URL tool for a fetch prompt → agent-multi-tool-selection.spec.ts
• agent selects the Web Search tool for a search prompt → agent-multi-tool-selection.spec.ts
• image via input handle is described by the agent → agent-multimodal-image-input.spec.ts
• negative control — no image, no image-specific description → agent-multimodal-image-input.spec.ts
• a small n_messages truncates retrieval to the most recent messages → agent-n-messages-limit.spec.ts
• causal control — a large n_messages retrieves the full seeded history → agent-n-messages-limit.spec.ts
• output_schema fields come back as typed JSON keys on the structured response → agent-structured-output.spec.ts
• a multiple (As List) schema row returns an array of the row's type → agent-structured-output.spec.ts
• Agent Instructions are respected in the model response → agent-system-prompt.spec.ts
• negative control — sentinel is absent without the instruction → agent-system-prompt.spec.ts
• agent handles a tool error and continues execution → agent-tool-error-handling.spec.ts
• an invalid tool name blocks execution with a clear message → agent-tool-name-validation.spec.ts
• causal control — a valid custom tool name executes normally → agent-tool-name-validation.spec.ts
• user must be able to send images in the playground with the agent component → general-bugs-agent-images-playground.spec.ts
• language model must respond with OpenAI provider → language-model-regression.spec.ts
• language model must respond with Google provider → language-model-regression.spec.ts
• language model provider switch from OpenAI to Google must persist → language-model-regression.spec.ts
• model provider dialog opens from the Language Model node → language-model-regression.spec.ts
• playground shows error when LLM run endpoint returns 500 (mocked invalid API key) → llm-invalid-api-key-ui.spec.ts
• playground input remains usable after API error (mocked) → llm-invalid-api-key-ui.spec.ts
• memory chatbot template loads with correct node structure → memory-history-regression.spec.ts
• message history context retention suite → memory-history-regression.spec.ts
• session isolation: new session has no context from previous session → memory-history-regression.spec.ts
• OpenAI provider is listed in Model Providers settings → model-provider-api-key.spec.ts
• Anthropic provider is listed in Model Providers settings → model-provider-api-key.spec.ts
• a configured provider exposes the key edit surface (Replace, no raw input) → model-provider-api-key.spec.ts
• page opens with its description and the available provider count → model-provider-modal-actions.spec.ts
• an invalid API key is rejected and does not enable the provider → model-provider-modal-actions.spec.ts
• selecting another provider switches the visible detail panel → model-provider-modal-actions.spec.ts
• model toggle changes immediately and persists across reopen → model-provider-model-toggle.spec.ts
• disabling a model removes it from a component model dropdown → model-provider-model-toggle.spec.ts
• the Language Model node renders its model selector → modelInputComponent.spec.ts
• opening the model dropdown lists model options → modelInputComponent.spec.ts
• the model dropdown exposes the Manage Model Providers entry → modelInputComponent.spec.ts
• the trigger shows the selected model name → modelInputComponent.spec.ts
• provider list renders with the known providers → modelProviderModal.spec.ts
• selecting a provider opens its API key configuration detail → modelProviderModal.spec.ts
• a configured provider shows its model selection panel → modelProviderModal.spec.ts
• should display error message when using invalid authentication for provider → provider-invalid-auth-error.spec.ts
• a provider credential variable can be removed through the Global Variables UI → remove-provider-api-key.spec.ts
• DELETE /api/v1/variables/{id} removes a provider API key variable → remove-provider-api-key.spec.ts

core-functionality/model-provider/

• Anthropic API key is configured via Settings → Model Providers → anthropic-provider.spec.ts
• configured Anthropic selects a Claude model in the Agent and executes the flow → anthropic-provider.spec.ts
• switches between Claude model families (Haiku → Sonnet → Opus) → anthropic-provider.spec.ts
• Azure AI Foundry is offered with a two-variable form and a Foundry-only deployment surface → azure-ai-foundry-provider-setup.spec.ts
• an unconfigured Azure AI Foundry panel is read-only: no enable toggle, no add-deployment control → azure-ai-foundry-provider-setup.spec.ts
• credentials that do not validate are rejected and nothing is persisted → azure-ai-foundry-provider-setup.spec.ts
• a portal deployment name absent from every catalog is accepted and rendered → azure-ai-foundry-provider-setup.spec.ts
• real credentials configure the provider and enable a portal deployment through the UI → azure-ai-foundry-provider-setup.spec.ts
• the configured deployment answers a real inference through the Language Model component → azure-ai-foundry-provider-setup.spec.ts
• Google API key is configured via Settings → Model Providers → google-provider.spec.ts
• configured Google selects a Gemini model in the Agent and executes the flow → google-provider.spec.ts
• Ollama base URL is configured via Settings → Model Providers → ollama-provider.spec.ts
• the provider is offered with two variables and a live-only, empty catalog → openai-compatible-provider-setup.spec.ts
• an unreachable base URL is rejected and nothing is persisted → openai-compatible-provider-setup.spec.ts
• a reachable endpoint with a bogus key is rejected as an authentication failure → openai-compatible-provider-setup.spec.ts
• the configured provider discovers exactly the models its endpoint serves → openai-compatible-provider-setup.spec.ts
• OpenAI API key is configured via Settings → Model Providers → openai-provider.spec.ts

core-functionality/observability-monitoring/

• a misconfigured flow surfaces an appropriate build-error message → flow-error-message.spec.ts
• Clearing traces for a flow whose trace has spans succeeds (cascade), leaving no traces behind → traces-delete-cascade.spec.ts
• DELETE /api/v1/monitor/traces returns 404 for an unknown flow_id → traces-delete.spec.ts
• DELETE /api/v1/monitor/traces?flow_id=... clears all traces, and a second DELETE on the empty owned flow still returns 204 → traces-delete.spec.ts
• GET /api/v1/monitor/traces/{trace_id} returns a populated tokenUsage + modelName on the LLM span → traces-detail-llm-span-populated.spec.ts
• GET /api/v1/monitor/traces/{trace_id} returns 404 for an unknown but well-formed UUID → traces-detail-single.spec.ts
• GET /api/v1/monitor/traces/{trace_id} returns the full TraceRead contract with a non-empty span tree → traces-detail-single.spec.ts
• GET /api/v1/monitor/transactions returns 200 with paginated result → traces-detail.spec.ts
• GET /api/v1/monitor/transactions filters by flow_id (UUID) → traces-detail.spec.ts
• transaction records contain required fields when not empty → traces-detail.spec.ts
• GET /api/v1/monitor/traces returns totalLatencyMs and totalTokens for a flow run → traces-latency-tokens.spec.ts
• Flow Activity page shows latency and token columns for the run → traces-latency-tokens.spec.ts
• Trace Details modal shows span tree and per-span latency → traces-latency-tokens.spec.ts
• GET /api/v1/monitor/traces?status=error returns only the failing trace; rejects unknown values → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?status=ok returns only the successful trace → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?start_time pins the >= lower bound → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?query= filters by trace name (incl. 50-char sanitize cap) → traces-list-filters.spec.ts
• GET /api/v1/monitor/traces?session_id filters by the session passed at run time → traces-list-filters.spec.ts
• should be able to see and interact with Traces → traces.spec.ts

core-functionality/playground/

• approving a Human Input pause routes only the approved branch → human-input-pause-resume.spec.ts
• rejecting a Human Input pause routes only the reject branch → human-input-pause-resume.spec.ts
• copy button copies Chat Input output and toggles Check icon → output-modal-copy-button.spec.ts
• playground must show one compact preview per attached image when two images are attached → playground-attachments-management.spec.ts
• playground must keep the remaining preview when one of two attachments is removed → playground-attachments-management.spec.ts
• playground must render both attached images in the user message after sending → playground-attachments-management.spec.ts
• playground input must return to empty state after removing the only attachment → playground-attachments-management.spec.ts
• playground swap flow must send only the second image when the first is removed before attaching the second → playground-attachments-management.spec.ts
• selecting an individual session checkbox must reveal the bulk-delete-button → playground-bulk-delete.spec.ts
• select-all-checkbox must select all non-default sessions → playground-bulk-delete.spec.ts
• bulk-delete-button must remove all selected sessions from the sidebar → playground-bulk-delete.spec.ts
• clear chat on Default session must remove messages but keep the session → playground-clear-history.spec.ts
• deleting a user-created session must remove it and return to Default session → playground-clear-history.spec.ts
• send button stays enabled regardless of input content → playground-empty-message-send.spec.ts
• clearing the input after typing leaves the field empty → playground-empty-message-send.spec.ts
• playground opens in fullscreen with chat input visible → playground-fullscreen.spec.ts
• playground closes and reopens correctly from the flow editor → playground-fullscreen.spec.ts
• messages sent in playground must persist after closing and reopening → playground-history-persist.spec.ts
• playground opens with chat textarea pre-filled from ChatInput Input Text → playground-input-text-prefill.spec.ts
• creating a new session re-applies the Input Text pre-fill → playground-input-text-prefill.spec.ts
• pre-filled value is sent as the first message of the session → playground-input-text-prefill.spec.ts
• edit user message — hover reveals edit button and saved changes replace original text → playground-message-edit.spec.ts
• cancel message edit — original text is preserved → playground-message-edit.spec.ts
• message edited in playground is reflected in Session Logs → playground-message-edit.spec.ts
• message-logs-option must open the Session Logs modal for the active session → playground-message-logs.spec.ts
• selecting messages in the log table and deleting them must reduce the row count → playground-message-logs.spec.ts
• playground must show non-image preview tile (delete button, no ) in input area after attaching a .txt file → playground-non-image-attachment.spec.ts
• playground must render non-image attachment in user message (truncated filename + zero file-images) after sending a .txt → playground-non-image-attachment.spec.ts
• playground must render JSON Data output as a code block → playground-output-data.spec.ts
• playground must render DataFrame output as a markdown table → playground-output-data.spec.ts
• playground must show image compact preview in input area after attaching an image → playground-output-image.spec.ts
• playground must display uploaded image in user message after sending → playground-output-image.spec.ts
• Playground run is delivered over an SSE (text/event-stream) response → playground-response-streaming-sse.spec.ts
• clear-chat removes all messages from Default Session → playground-session-clear.spec.ts
• a session renamed in the playground is the session its messages are stored under → playground-session-id.spec.ts
• new-chat button must add a new session entry to the sidebar → playground-session-nav.spec.ts
• session selector sidebar must switch to the selected session → playground-session-nav.spec.ts
• rename option must not be available for the Default Session → playground-session-rename.spec.ts
• rename option must not be available for a session with no messages → playground-session-rename.spec.ts
• rename option must be available and functional for a session with messages → playground-session-rename.spec.ts
• Shareable playground URL is generated when publishing is enabled → playground-shareable-url.spec.ts
• user message must appear instantly in playground before AI responds → playground-ux.spec.ts
• playground must scroll to latest message after sending → playground-ux.spec.ts
• playground input field must be ready after flow responds → playground-ux.spec.ts
• User must be able to stop building from inside Playground → stop-button-playground.spec.ts

core-functionality/project-management/

• user should be able to select flows with different methods and perform bulk actions → bulk-actions.spec.ts
• user should be able to edit flow name and see it reflected in the main page listing → edit-flow-name.spec.ts
• flow settings enforce character limits and persist name & description → flowSettings.spec.ts
• creates, renames and deletes an empty project folder via the UI → folder-crud.spec.ts
• deleting a folder that contains a flow removes the flow with it → folder-crud.spec.ts
• deleting a folder should update the folder list immediately → folder-deletion-integrity.spec.ts
• deleting one folder should not affect other folders → folder-deletion-integrity.spec.ts
• creating a new folder after deletion should work correctly → folder-deletion-integrity.spec.ts
• getting-started progress increments as onboarding steps complete → user-progress-track.spec.ts

flow-functionality/

• API access modal opens from the Publish dropdown exposing the Python, JavaScript and cURL tabs → api-access-modal-regression.spec.ts
• API access modal switches the displayed snippet when changing language tabs → api-access-modal-regression.spec.ts
• API access modal embeds the current flow ID in the generated run endpoint URL → api-access-modal-regression.spec.ts
• API access modal closes cleanly via Escape and via the close button → api-access-modal-regression.spec.ts
• user should be able to manually save a flow when the auto_save is off → auto-save-off.spec.ts
• connecting two compatible components creates exactly one edge → canvas-connect-components.spec.ts
• connecting the same compatible pair twice does not duplicate the edge → canvas-connect-components.spec.ts
• a type-incompatible pair does not connect → canvas-connect-components.spec.ts
• clicking the same target handle twice does not create an edge → canvas-connect-components.spec.ts
• copy and paste ChatOutput component via Ctrl+C / Ctrl+V → canvas-copy-paste.spec.ts
• copy and paste Prompt Template (component with dynamic ports) via Ctrl+C / Ctrl+V → canvas-copy-paste.spec.ts
• clicking empty canvas area deselects a selected node → canvas-deselect-node.spec.ts
• pressing Escape deselects a selected node → canvas-deselect-node.spec.ts
• deleting an edge from its context menu removes it from the canvas and the flow → canvas-edge-reconnect.spec.ts
• an edge can be recreated after it is deleted → canvas-edge-reconnect.spec.ts
• dragging a component moves it on the canvas and persists the new position → canvas-move-node.spec.ts
• a Shift+drag marquee selects every component it encloses → canvas-multiselect.spec.ts
• deleting a box selection clears the selected components → canvas-multiselect.spec.ts
• deleting a sticky note from its options menu removes it everywhere → canvas-sticky-note-delete.spec.ts
• deleting a sticky note with Backspace removes it everywhere → canvas-sticky-note-delete.spec.ts
• deleting one of two sticky notes leaves the other in place → canvas-sticky-note-delete.spec.ts
• user can create a blank flow from the new-project modal → create-blank-flow.spec.ts
• user can create a flow from a starter template → create-flow-from-template.spec.ts
• user can copy a valid macOS/Linux curl command from the API access modal → curlApiGeneration.spec.ts
• user can duplicate a flow from the home page dropdown menu → duplicate-flow.spec.ts
• duplicate flow via API auto-suffixes the name on collision → duplicate-flow.spec.ts
• export flow to JSON triggers success toast and produces a valid file → export-import-flow.spec.ts
• imported JSON flow must load all components on canvas → export-import-flow.spec.ts
• import flow from JSON via upload button must load flow on canvas → export-import-flow.spec.ts
• 1 - runs the flow from the canvas terminal node → flow-execution-canvas.spec.ts
• 2 - the flow ran correctly: every node reached build success → flow-execution-canvas.spec.ts
• 3 - the chat input and chat output are visible in the Playground → flow-execution-canvas.spec.ts
• should lock and unlock a flow and verify UI changes → flow-lock.spec.ts
• should show correct lock/unlock icon in settings based on state → flow-lock.spec.ts
• flow can be renamed via the header edit → flow-rename-header.spec.ts
• flow name persists after rename via API PATCH and GET → flow-rename-header.spec.ts
• import invalid JSON must show error message → import-invalid-json.spec.ts
• import non-JSON file must show error message → import-invalid-json.spec.ts
• import JSON with missing data field must show error → import-invalid-json.spec.ts
• user must be able to lock a flow and it must be saved → lock-flow.spec.ts
• user can publish a flow and access it via shareable URL, then unpublish to revoke access → publish-flow.spec.ts
• publish flow via API toggles access_type between PUBLIC and PRIVATE → publish-flow.spec.ts
• user can copy a valid Python requests snippet from the API access modal → pythonApiGeneration.spec.ts
• flow state should be properly cleaned up between user sessions → user-flow-state-cleanup.spec.ts

mcp/client/

• Gemini invokes the echo MCP tool (regression for fixed upstream #440) → mcp-client-agent-gemini-tool-regression.spec.ts
• configures MCP server via JSON, selects echo tool, runs it, and verifies output → mcp-client-regression.spec.ts
• configures MCP server via HTTP form tab and verifies registration → mcp-client-regression.spec.ts
• selects get-sum tool, provides numeric inputs, and verifies sum in output → mcp-client-regression.spec.ts
• registering an already-existing MCP server returns 409 Conflict → mcp-server-registration-status-codes.spec.ts
• deleting a non-existent MCP server returns 404 Not Found → mcp-server-registration-status-codes.spec.ts

mcp/server/

• generated endpoint advertises the project and lists the enabled flow → mcp-server-protocol.spec.ts
• execute the exposed tool over the MCP protocol echoes the input → mcp-server-protocol.spec.ts
• resources/list surfaces the uploaded flow file as a resource → mcp-server-resources.spec.ts
• user must be able to see starter projects for mcp servers → mcp-server-starter-projects.spec.ts
• user must not be able to add duplicate mcp servers from starter projects → mcp-server-starter-projects.spec.ts
• user must be able to add and delete MCP server from sidebar → mcp-server.spec.ts
• STDIO MCP server fields should persist after saving and editing → mcp-server.spec.ts
• HTTP/SSE MCP server fields should persist after saving and editing → mcp-server.spec.ts
• mcp server tools should be refreshed when editing a server → mcp-server.spec.ts
• stdio command with an embedded argument is refused, and command plus args is accepted → mcp-server.spec.ts

ui-ux/

• select and delete a flow → actionsMainPage-shard-1.spec.ts
• serializes created_at/expires_at with UTC offset and no microseconds → api-keys-timezone-display.spec.ts
• renders API key timestamps in the viewer's local timezone → api-keys-timezone-display.spec.ts
• zoom in and zoom out step the canvas scale and clamp at the React Flow bounds → canvas-zoom-navigation.spec.ts
• Fit View centers every node inside the canvas viewport → canvas-zoom-navigation.spec.ts
• Fit View is reachable from the canvas controls toolbar → canvas-zoom-navigation.spec.ts
• wheel scroll navigates the canvas anchored at the pointer → canvas-zoom-navigation.spec.ts
• user can edit the text of an existing sticky note and the canvas reflects only the new text → edit-sticky-note-text.spec.ts
• executing flow with network error shows error feedback → execution-error-notification.spec.ts
• executing flow with server error shows error feedback → execution-error-notification.spec.ts
• user must see on handle click the possibility connections → filterSidebar.spec.ts
• create a Generic global variable from Settings page → global-variable-edit.spec.ts
• create a Generic type global variable → global-variables-crud.spec.ts
• delete a global variable removes it from the list → global-variables-crud.spec.ts
• Credential variable value is hidden from the variable list → global-variables-crud.spec.ts
• user can search and add components using keyboard shortcuts → keyboardComponentSearch.spec.ts
• LangflowShortcuts → langflowShortcuts.spec.ts
• the main menu lists every item, reports the running version and links out → main-menu-actions.spec.ts
• the main menu's Settings action navigates to the Settings page → main-menu-actions.spec.ts
• user must be able to minimize and expand a component → minimize.spec.ts
• User should be able to interact notifications tab → notifications.spec.ts
• right-clicking a component selects it and opens its options menu → right-click-dropdown.spec.ts
• an item picked from the right-click menu acts on that component → right-click-dropdown.spec.ts
• right-clicking the canvas background opens no menu and dismisses an open one → right-click-dropdown.spec.ts
• Settings General section loads and shows its header → settings-general-section.spec.ts
• Settings Messages section is accessible → settings-general-section.spec.ts
• Settings Shortcuts section is accessible and lists shortcuts → settings-general-section.spec.ts
• Settings > Messages displays sent messages in correct order with working filters → settings-message-history.spec.ts
• user can access Settings page from the profile menu → settings-navigation.spec.ts
• Settings page shows all main sections in sidebar navigation → settings-navigation.spec.ts
• Settings Shortcuts section lists keyboard shortcuts → settings-navigation.spec.ts
• Settings Model Providers section loads with provider configuration → settings-navigation.spec.ts
• dark and light mode toggle correctly updates the body class → settings-theme-toggle.spec.ts
• double-click on a sidebar component adds it to the canvas → sidebar-add-component.spec.ts
• dragging a sidebar component drops the node at the pointer → sidebar-add-component.spec.ts
• an added component arrives with its catalog default settings → sidebar-add-component.spec.ts
• searching by name lists the matching component and hides the others → sidebar-search-and-filter.spec.ts
• a query with no match shows the empty state and clearing restores the tree → sidebar-search-and-filter.spec.ts
• a provider query groups its components under the provider bundle → sidebar-search-and-filter.spec.ts
• category disclosures collapse and expand their component list → sidebar-search-and-filter.spec.ts
• adding a sticky note places it on the canvas and in the flow → sticky-notes.spec.ts
• changing a sticky note colour repaints it and persists the choice → sticky-notes.spec.ts
• resizing a sticky note grows it and persists the new size → sticky-notes.spec.ts

---

🔵 Phase 1 — Next Delivery

Validate ([-]) and create ([ ]) in the modules below. See details in Part II.

Module	Validate ([-])	Create ([ ])
api/flows/ — REST API	0	0
core-components/ — Component Config	3	0
core-components/ — Core Components	3	1
core-functionality/auth/	13	0
core-functionality/llm-agents/	5	2
core-functionality/model-provider/	2	0
core-functionality/playground/	3	1
mcp/client/	1	2
mcp/server/	1	1
ui-ux/ — Canvas	0	0

---

🟡 Phase 2 — Next Delivery

Remaining modules after Phase 1 completion. See details in Part II.

Module	Validate ([-])	Create ([ ])
core-functionality/observability-monitoring/	0	0
core-functionality/knowledge-ingestion/	0	0
flow-functionality/	1	0
core-functionality/project-management/	6	0
core-functionality/templates/	39	0
ui-ux/ — Settings	0	0