Langflow — Test Scenario Guide (Step by Step)

Generated from QA-CHECKLIST.md to facilitate understanding and manual validation of tests.

Status legend:

[-] → automated, needs validation

[x] → automated and validated

[ ] → needs to be created

[~] → partially covered

[!] → flaky / unstable

Index

REST API — Health Check
REST API — Flow CRUD
REST API — Flow Execution
REST API — Components and Messages
REST API — Integration Code Generation
Component Configuration — Parameters Panel
Tool Mode
Component Updates
Core Components — Chat Input/Output
Core Components — Prompt Template
Core Components — API Request
Core Components — Webhook
Core Components — Agent (13.1–13.13 — includes new provider, memory, tools and output scenarios)
Authentication — Login and Logout
User Management (Admin)
Global Variables (API Keys)
File Upload and Processing
LLM Agents — Execution and Control (18.1–18.17 — includes context_id, multi-tool, tool error)
Model Providers
Observability — Traces and Notifications
Playground — Chat and Session
Project and Folder Management
Templates and Starter Projects
Flow — CRUD and Operations
MCP — Client and Server
UI/UX — Sidebar and Canvas
Core Components — Loop
API Keys — Timestamps & Expiry
UI/UX — Settings Shortcuts
Core Components — Nested / Grouping

1. REST API — Health Check

File: core/features/api-health-check.spec.ts

1.1.a GET `/health_check` → status 200 `[-]`

Objective: Confirm that the Langflow server is online and healthy.

Precondition: Langflow running at http://localhost:7860.

Step by step:

Make a GET /health_check request without authentication.
Verify that the returned HTTP status is 200.
Verify that the response body contains { status: "ok", db: "ok" }.

Validation: Server responded with 200 and database is accessible.

1.1.b GET `/api/v1/version` → returns version, main_version, package `[x]`

Objective: Verify that the version endpoint returns instance metadata.

Step by step:

Make a GET /api/v1/version request without authentication.
Verify that the status is 200.
Verify that the body contains version, main_version and package fields.
Make a POST /api/v1/version request; verify that the status is 405.

Validation: Version information is present; package is Langflow. POST is rejected with 405.

2. REST API — Flow CRUD

File: core/features/api-flows-crud.spec.ts

2.1 POST `/api/v1/flows/` → creates flow, returns ID `[-]`

Objective: Verify that a flow can be created via API.

Precondition: Authentication token obtained via /api/v1/auto_login.

Step by step:

Get Bearer token via GET /api/v1/auto_login.
Make POST /api/v1/flows/ with body { name, description, data: { nodes: [], edges: [] }, is_component: false }.
Verify that the returned status is 201 Created.
Verify that the response body contains an id field (UUID).

Validation: Flow created with unique ID returned.

2.2 GET `/api/v1/flows/` → lists user flows `[-]`

Objective: Confirm that the flow listing returns the authenticated user's flows.

Step by step:

Create at least one flow via POST (setup).
Make GET /api/v1/flows/ with header Authorization: Bearer <token>.
Verify that the status is 200.
Verify that the returned array contains the created flow (by ID or name).

Validation: List includes the created flow and does not include other users' flows.

2.3 GET `/api/v1/flows/{id}` → returns flow by ID `[-]`

Objective: Confirm that a specific flow is returned correctly by its ID.

Step by step:

Create flow and save the returned id.
Make GET /api/v1/flows/{id} with Bearer header.
Verify that the status is 200.
Verify that the id field in the response equals the requested ID.

Validation: Returned flow corresponds to the queried ID.

2.4 PATCH `/api/v1/flows/{id}` → updates name/description `[-]`

Objective: Verify that flow fields can be updated via API.

Step by step:

Create flow and save the id.
Make PATCH /api/v1/flows/{id} with body { name: "New Name" }.
Verify that the status is 200.
Make GET /api/v1/flows/{id} and confirm that the name was updated.

Validation: Name/description reflect the values sent in the PATCH.

2.5 DELETE `/api/v1/flows/{id}` → removes flow, returns 200 `[-]`

Objective: Confirm that a flow can be deleted via API.

Step by step:

Create flow and save the id.
Make DELETE /api/v1/flows/{id} with Bearer header.
Verify that the status is 200.

Validation: Deletion confirmed with status 200.

2.6 GET after DELETE → should return 404 `[-]`

Objective: Ensure that a deleted flow is no longer accessible.

Step by step:

Delete a flow by ID.
Make GET /api/v1/flows/{id} with the same deleted ID.
Verify that the returned status is 404 Not Found.

Validation: Attempt to access a deleted flow results in 404.

3. REST API — Flow Execution

Files: core/features/api-run-flow.spec.ts, api-run-with-tweaks.spec.ts

3.1 POST `/api/v1/run/{flow_id}` with `input_value` `[-]`

Objective: Execute a flow via API and receive a response.

Precondition: Flow created and API key generated via /api/v1/api_key/.

Step by step:

Get Bearer token via auto_login.
Create API key via POST /api/v1/api_key/ → save api_key and id.
Create test flow.
Make POST /api/v1/run/{flow_id} with header x-api-key: <key> and body { input_value: "Hello", input_type: "chat", output_type: "chat" }.
Verify that the status is 200.
Verify that the body contains outputs.

Validation: Flow executed with structured response in outputs.

3.2 POST with `tweaks` → parameters override configuration `[-]`

Objective: Verify that tweaks override parameters configured in the flow.

Step by step:

Create flow with a configured component.
Make POST /api/v1/run/{flow_id} including field tweaks: { "ComponentName": { "parameter": "value" } }.
Verify that the execution uses the tweak value (not the original from the flow).

Validation: The parameter overridden by tweak is used in the execution.

3.3 POST with custom `session_id` `[-]`

Objective: Ensure that custom sessions isolate conversation history.

Step by step:

Execute flow with session_id: "session-abc" in the body.
Verify that the session_id returned in the response matches the one sent.
Execute again with session_id: "session-xyz" and verify it is an independent session.

Validation: Each session_id maintains isolated history.

3.4 POST with invalid API key → returns 401/403 `[-]`

Objective: Confirm that the API protects execution with authentication.

Step by step:

Make POST /api/v1/run/{flow_id} with header x-api-key: invalid-key.
Verify that the returned status is 401 or 403.

Validation: Access denied with invalid credential.

3.5 POST to non-existent flow → returns 404 `[-]`

Objective: Confirm behavior when flow does not exist.

Step by step:

Make POST /api/v1/run/00000000-0000-0000-0000-000000000000 with valid API key.
Verify that the status is 404.

Validation: Endpoint returns 404 for non-existent flow.

4. REST API — Components and Messages

4.1 GET `/api/v1/all` → lists available components `[-]`

Objective: Verify that the component catalog is accessible.

Step by step:

Make GET /api/v1/all with Bearer header.
Verify that the status is 200.
Verify that the body is an object with multiple keys (component names).

Validation: Catalog returns all registered components.

4.2 GET `/api/v1/monitor/messages` → returns array `[-]`

Objective: Verify that the message history of a flow is accessible via API.

Precondition: Flow executed at least once to generate messages.

Step by step:

Execute flow and save the flow_id (UUID).
Make GET /api/v1/monitor/messages?flow_id={uuid} with Bearer.
Verify that the status is 200.
Verify that the body is an array.

Validation: Endpoint returns 200 and array of messages.

⚠️ flow_id must be a valid UUID — arbitrary strings return 422.

4.3 GET with session_id filter `[-]`

Objective: Verify that messages can be filtered by session.

Step by step:

Execute flow with session_id: "test-session".
Make GET /api/v1/monitor/messages?flow_id={uuid}&session_id=test-session.
Verify that all returned messages belong to the filtered session.

Validation: session_id filter isolates messages from the correct session.

5. REST API — Integration Code Generation

5.1 Generate curl for execution `[x]`

Objective: Verify that Langflow generates a valid curl command for flow execution.

Step by step:

Open a flow in the editor.
Click the "API Access" button (api-access-button).
Select the cURL tab.
Verify that the generated code targets the correct flow URL and uses the POST HTTP method (the actual curl invocation may use either -X POST or --request POST depending on Langflow's snippet generator).

Validation: Generated curl code points to the correct flow endpoint.

5.2 Generate Python code for integration `[x]`

Objective: Verify that Langflow generates functional Python code to call the flow.

Step by step:

Open a flow in the editor.
Click the "API Access" button.
Select the Python tab.
Verify that the code contains import requests and the flow URL.

Validation: Generated Python code contains the correct execution parameters.

6. Component Configuration — Parameters Panel

Files: core/unit/inputComponent.spec.ts, dropdownComponent.spec.ts, etc.

6.1 Open component advanced options `[-]`

Objective: Verify that the advanced parameters panel can be opened.

Step by step:

Add any component to the canvas.
Click "Advanced" or the component settings icon.
Verify that the advanced options panel expands showing additional fields.

Validation: Advanced options panel visible with configurable fields.

6.2 Edit text field (input) `[-]`

Step by step:

Add component with a text field (e.g.: Chat Input).
Click on the text field of the component.
Type a value (e.g.: "my text").
Verify that the value was saved in the field.

Validation: Text field displays the typed value.

6.3 Edit dropdown `[-]`

Step by step:

Add component with dropdown (e.g.: OpenAI with model selection).
Click the dropdown.
Select a different option from the default.
Verify that the selected option is visible in the dropdown.

Validation: Dropdown reflects the selected option.

6.4 Edit toggle `[-]`

Step by step:

Add component with toggle (e.g.: "Stream" option).
Check the initial state of the toggle (on/off).
Click the toggle to invert the state.
Verify that the toggle changed state.

Validation: Toggle correctly alternates between states.

6.5 Edit float / int field `[-]`

Step by step:

Locate a numeric field in a component (e.g.: Temperature = 0.7).
Click the field and change the value.
Press Enter or click elsewhere.
Verify that the value was updated.

Validation: Numeric field accepts and displays the new value.

6.6 Edit slider `[-]`

Step by step:

Locate a component with a slider (e.g.: temperature parameter in slider mode).
Drag the slider to the right or left.
Verify that the corresponding numeric value is updated.

Validation: Slider and numeric value are synchronized.

7. Tool Mode

Files: extended/features/tool-mode.spec.ts, core/features/toolModeGroup.spec.ts

7.1 Enable Tool Mode on a component `[-]`

Objective: Verify that a component can be enabled as a "Tool" for use by Agents.

Step by step:

Add component to canvas (e.g.: API Request).
Locate the "Tool Mode" toggle in the component.
Click to enable.
Verify that the component displays a visual indication of active Tool Mode.
Verify that the tool handle becomes available for connection with an Agent.

Validation: Component in Tool Mode displays tool handle and visual indication.

7.2 Group components in Tool Mode `[-]`

Step by step:

Add two or more components to the canvas in Tool Mode.
Select all components (Shift+drag).
Use the grouping option (context menu or shortcut).
Verify that the group maintains the Tool Mode settings.

Validation: Created group preserves the Tool Mode of internal components.

8. Component Updates

Files: (not yet implemented)

8.1 Outdated component notification `[-]`

Objective: Ensure that Langflow alerts the user when a component is on an old version.

Step by step:

Import a flow JSON that contains an outdated component version.
Open the flow in the editor.
Verify that the component displays an "outdated" icon/badge or notification.
Verify that there is an option to update the component.

Validation: Outdated component badge visible with update action available.

8.2 Update component action `[-]`

Step by step:

Identify component with outdated badge.
Click the "Update" option of the component.
Verify that the outdated badge disappears.
Verify that the component maintains its settings after the update.

Validation: Component updated without loss of configuration.

9. Core Components — Chat Input/Output

Files: core/unit/chatInputOutput.spec.ts, core/integrations/textInputOutput.spec.ts

9.1 ChatInput receives user message `[-]`

Objective: Verify that the Chat Input component processes an input message.

Step by step:

Create flow with Chat Input component connected to Chat Output.
Open Playground (button playground-btn-flow-io).
Type a message in the input-chat-playground field.
Click the send button (button-send).
Verify that the user message appears in the chat history.

Validation: Sent message appears in the Playground interface.

9.2 ChatOutput displays LLM response `[-]`

Step by step:

Create flow: Chat Input → LLM (OpenAI/Anthropic) → Chat Output.
Configure API key in the LLM component.
Open Playground and send a message.
Wait for LLM response.
Verify that the response appears in the chat box (div with assistant message).

Validation: LLM response displayed in Playground after execution.

10. Core Components — Prompt Template

Files: core-components/prompt-template-component-regression.spec.ts

10.1 Prompt with variables in curly braces `[x]`

Objective: Verify that {name} variables in the prompt create dynamic handles.

Step by step:

Add Prompt Template component to canvas.
Click "Edit Prompt" (button button_open_prompt_modal).
In the modal text field, type: Hello {name}, your role is {role}.
Click "Save" (genericModalBtnSave).
Verify that two handles were created: name and role (left side of the component).

Validation: Handles handle-prompt template-shownode-name-left and handle-prompt template-shownode-role-left visible.

10.2 Removing variable from prompt deletes corresponding port `[x]`

Step by step:

Create prompt with variable {name} (handle name created).
Reopen the prompt modal and remove {name} from the text.
Save.
Verify that the name handle disappeared from the component.

Validation: Handle removed when variable is deleted from prompt.

11. Core Components — API Request

Files: core-components/api-request-component-regression.spec.ts

11.1 Renders on canvas with URL and API Response handles `[x]`

Step by step:

Add "API Request" component to canvas.
Verify the node title title-API Request is visible.
Verify both handle-apirequest-shownode-api response-right (output) and handle-apirequest-shownode-url-left (input) handles render.

Validation: Exactly one node on the canvas with both URL input and API Response output handles visible.

11.2 Inspector accepts URL and HTTP method values `[x]`

Step by step:

Add API Request component.
Fill popover-anchor-input-url_input with https://httpbin.org/get and confirm the value persists.
Open the method dropdown, select POST, and confirm the displayed value updates.

Validation: URL and method fields reflect the configured values; no validation errors.

11.3 Invalid URL is accepted by field but run shows error notification `[x]`

Step by step:

Call allowFlowErrors() (the run is expected to fail).
Add the component, fill not-a-url in the URL field.
Run the component.
Assert the toast Error building Component API Request: and the detail Invalid URL provided: are visible.

Validation: The component does not crash on an invalid URL; the toast surfaces the descriptive error and the run button remains usable.

11.4–11.8 Execute GET / POST / PUT / PATCH / DELETE returns 200 `[x]`

Step by step (per verb):

Add API Request component.
Fill the URL with https://httpbin.org/<verb> (each endpoint only accepts that verb — any other returns 405).
Select the matching method in the dropdown.
Run the component.
Open the output via output-inspection-api response-apirequest.
Assert the output Data contains 200, the echoed URL, status_code, response_headers, and result.

Validation: Each verb correctly executes and returns 200 with the structural Data fields populated.

11.9 Non-2xx HTTP response (404) propagates as status_code without crashing `[x]`

Step by step:

Fill https://httpbin.org/status/404.
Run the component.
Assert the output Data contains 404, the source key, and does not contain an "error" field.

Validation: A 404 response is propagated as status_code: 404 (not surfaced as an exception). The error field appears only on httpx transport exceptions.

11.10 Query parameters embedded in URL are sent and echoed `[x]`

Step by step:

Fill https://httpbin.org/get?e2e_param=functional_test_value.
Run the component.
Assert the output contains the parameter key (e2e_param) and value (functional_test_value) and status 200.

Validation: Query parameters in the URL are forwarded and echoed by httpbin.

11.11 Headers table accepts key + value cell entries via inspector `[x]`

Step by step:

Open the headers table via div-table_headers → Open table button.
Add a row, fill the [col-id="key"] cell with X-E2E-Header and the [col-id="value"] cell with test-header-value via the inline View Text editor.
Each fillViewTextCell call asserts the saved cell value renders as a button inside the table dialog.
Close with btn-cancel-modal; verify canvas integrity.

Validation: Both key and value cells accept text input through the View Text editor and render the saved value in-session.

11.12 cURL tab switches mode and field accepts a cURL command `[x]`

Step by step:

Switch to the cURL tab via tab_1_curl.
Fill textarea_str_curl_input with a valid cURL command (curl -X GET ... -H 'Accept: application/json').
Assert the cURL handle handle-apirequest-shownode-curl-left is visible.

Validation: The cURL tab is reachable, the textarea accepts the command, and the cURL handle is exposed on the node.

11.13 cURL parser auto-fills URL field and executes the GET, returning 200 `[x]`

Step by step:

Switch to the cURL tab before touching url_input (pre-filling would mask a parser regression).
Fill the cURL command with the URL embedded.
Wait for url_input to be auto-populated by the parser (waitForFunction polls document.getElementById('popover-anchor-input-url_input').value).
Run the component and assert the output Data contains 200, the echoed URL, status_code, and result.

Validation: The cURL parser extracts the URL from the command and feeds the run end-to-end.

12. Core Components — Webhook

Files: core-components/webhook-component-regression.spec.ts

12.1 Webhook component displayed on canvas `[-]`

Step by step:

Search "Webhook" in the sidebar.
Add to canvas via double-click or drag.
Verify that the component appears on the canvas with its default settings.

Validation: Webhook component visible on canvas with handles and default settings.

12.2 Webhook URL generated automatically `[-]`

Step by step:

Add Webhook component to canvas.
Verify that the webhook URL field is populated automatically.
Confirm that the URL contains the flow ID (format /api/v1/webhook/{flow_id}).

Validation: Webhook URL automatically generated with correct flow ID.

12.3 POST endpoint accepts JSON and plain-text bodies returning 202 `[x]`

Objective: Confirm that POST /api/v1/webhook/{flowId} accepts both application/json and text/plain bodies and returns 202 with {status: "in progress", message: "Task started in the background"}.

Preconditions:

A blank flow with the Webhook component on the canvas (created via UI; autosave persists the flow).
A temporary x-api-key is required because Langflow's WEBHOOK_AUTH_ENABLE defaults to True since 1.9.2+ (PR langflow-ai/langflow#12845).

Step by step:

Add the Webhook component to a blank flow via the sidebar.
Wait for autosave (4 s debounce) before any webhook POST.
Create a temporary API key via POST /api/v1/api_key/.
POST a JSON object with the x-api-key header to /api/v1/webhook/{flowId}.
POST a plain-text body with x-api-key and Content-Type: text/plain to the same endpoint.
Delete the temporary API key in finally.

Validation:

JSON POST returns 202 with status === "in progress" and message === "Task started in the background".
Plain-text POST returns 202 with status === "in progress".
The temporary API key is deleted regardless of test outcome.

13. Core Components — Agent

Files: core-components/agent-component-regression.spec.ts (canvas rendering and provider field plumbing), agent-reasoning-steps.spec.ts, agent-system-prompt.spec.ts, agent-model-connection-isolation.spec.ts, agent-config-persistence.spec.ts, agent-max-iterations.spec.ts, agent-max-tokens.spec.ts, agent-reasoning-effort.spec.ts, agent-input-sources.spec.ts, agent-structured-output.spec.ts, agent-empty-refusal-response.spec.ts, agent-current-date-tool.spec.ts, agent-parse-error-behavior.spec.ts, agent-multimodal-image-input.spec.ts

13.1 Agent component displayed on canvas with default settings `[x]`

File: core-components/agent-component-regression.spec.ts

Step by step:

Open a blank flow and drag the Agent component from the "models & agents" disclosure to the canvas.
Verify that the node shows the title (title-Agent) and the three core handles:
- Input handle for "Language Model" (handle-agent-shownode-language model-left)
- Input handle for "Tools" (handle-agent-shownode-tools-left)
- Output handle "Response" (handle-agent-shownode-response-right)
Verify that the System Prompt field (textarea_str_system_prompt) and the model dropdown (value-dropdown-model_model) are visible in the default rendered state.

Validation: Agent component renders with title, all three core handles, and the default fields visible.

Same spec — core-components/agent-component-regression.spec.ts — also covers (canvas-level, distinct from the execution scenarios below):

System prompt persistence: typing a system prompt autosaves; after navigating away and reopening the flow by name, the value is restored.

Model dropdown entry point: opening value-dropdown-model_model surfaces the manage-model-providers button (the centralized provider-config path in 1.11) and lists the configured providers' models, each tagged with its icon-{Provider}.

Provider-icon swap: selecting a model from a different provider (OpenAI → Anthropic) replaces the icon-{Provider} mark on the Agent node trigger. Self-skips when both providers are not pre-configured in the local instance.

13.2 Connecting an external model in Agent drops the prior model selection `[x]`

File: agent-model-connection-isolation.spec.ts

Objective: Ensure connection-mode isolation in the Langflow 1.11 unified model picker: choosing "Connect other models" clears the previously selected model (and the node's secret fields), so a stale provider configuration cannot reach a backend run. The 1.11 Agent has no inline per-provider credential fields (credentials are global, under Settings → Model Providers), so the older "switch provider → provider-specific fields disappear" scenario no longer applies.

Step by step:

Load "Simple Agent" template with a configured provider/model (resolved from models.json / MODEL_TEST_ID).
Confirm the model picker (model_model) shows a concrete model name in value-dropdown-model_model (not the "Select a model" placeholder).
Open the picker and click the connect-other-models footer button.
Verify value-dropdown-model_model now reads "Connect other models" — the connection-mode label that replaces the prior model selection.

Validation: The previously selected model is dropped and the trigger reflects connection mode; the prior provider selection cannot leak into execution.

13.3 Flow with Agent saved and reopened — settings persist `[ ]`

File: agent-config-persistence.spec.ts

Objective: Confirm that when saving a flow with a configured Agent and reopening it, all parameters are preserved.

Step by step:

Load "Simple Agent" template with provider X, model Y.
Configure Agent Instructions = "You are an assistant specialized in Python.".
Configure Max Iterations = 5.
Save the flow (auto-save or Ctrl+S).
Navigate to the main page and open another flow.
Return to the original flow.
Verify that provider, model, Agent Instructions and Max Iterations have the configured values.

Validation: All Agent settings are preserved after saving and reopening the flow.

13.4 max_iterations limits agent cycles `[ ]`

File: agent-max-iterations.spec.ts

Objective: Verify that the max_iterations parameter is respected and the agent stops when the limit is reached.

Step by step:

Load "Simple Agent" template with a tool connected (e.g.: Calculator).
Configure Max Iterations = 1 in the Agent component.
Send a prompt that would normally require multiple cycles (e.g.: "Calculate 5+3 and then multiply by 2").
Wait for execution to finish.
Verify that the agent responded (did not fail silently).
Verify in "Agent Steps" that there was at most 1 reasoning iteration.

Validation: Agent stops after 1 iteration and returns response or limit-reached message.

13.5 max_tokens limits response size `[ ]`

File: agent-max-tokens.spec.ts

Objective: Verify that the max_tokens parameter is included in the payload sent to the model API.

Step by step:

Intercept requests to **/api/v1/run/** via page.route.
Load "Simple Agent" template and configure Max Tokens = 50.
Send a prompt that normally generates a long response (e.g.: "Write 500 words about AI").
Verify in the intercepted payload that max_tokens: 50 is present.
Verify that the response is shorter than without the limit.

Validation: max_tokens parameter present in the payload and response truncated as expected.

13.6 reasoning_effort field is conditional on the model `[ ]`

File: agent-reasoning-effort.spec.ts

Objective: Verify that the reasoning effort field only appears for models that support this feature.

Step by step:

Load "Simple Agent" template with a model that supports reasoning (e.g.: claude-sonnet-4-5).
Verify if reasoning_effort field is visible in the Agent component.
Switch to a model that does not support reasoning (e.g.: gpt-4o-mini).
Verify that the reasoning_effort field is not visible (or is disabled).

Validation: reasoning_effort field appears/disappears based on the capability of the selected model.

13.7 Agent Instructions (system prompt) is respected `[ ]`

File: agent-system-prompt.spec.ts

Objective: Confirm that the content of the Agent Instructions field is sent as a system prompt and influences the model's response.

Precondition: Provider with valid API key configured.

Step by step:

Load "Simple Agent" template.
Configure Agent Instructions = "Always respond in French, regardless of the question language.".
Open Playground.
Send message in English: "What is the capital of France?".
Wait for response.
Verify that the response is in French.

Scenario B — Empty system prompt:

Clear the Agent Instructions field.
Send any message.
Verify that the agent responds normally (no crash or error).

Validation: System prompt influences response; empty field does not cause failure.

13.8 Input via direct field vs handle (ChatInput) `[ ]`

File: agent-input-sources.spec.ts

Objective: Verify that the Agent accepts input both from the input_value field directly and via a handle connected to ChatInput.

Step by step (Scenario A — direct field):

Add Agent to canvas without ChatInput connected.
Fill input_value field directly: "Hello from direct input".
Click Run on the component.
Verify that the response is generated.

Step by step (Scenario B — via handle):

Load "Simple Agent" template (ChatInput connected to Agent).
Open Playground and send a message.
Verify that the message reaches the Agent and the response is returned.

Validation: Both input forms work correctly.

13.9 Structured output schema generates valid JSON `[ ]`

File: agent-structured-output.spec.ts

Objective: Verify that when output_schema is configured, the Agent returns JSON with the defined fields.

Step by step:

Load "Simple Agent" template.
Open Agent advanced settings.
Configure output_schema with fields: name (string), age (integer).
Configure format_instructions = "Respond with a JSON object with 'name' and 'age' fields.".
Send prompt: "Generate a fictional person named John who is 30 years old.".
Wait for response.
Verify that the response contains JSON with name and age fields.

Validation: Valid JSON returned with the fields from the configured schema.

13.10 Empty response or model refusal — no crash `[ ]`

File: agent-empty-refusal-response.spec.ts

Objective: Verify that the Agent component does not crash when the model refuses or returns an empty response.

Step by step (via mock):

Intercept the LLM API call via page.route.
Return an empty response (body "", status 200).
Send a message in the Playground.
Verify that the Playground does not freeze — some message is displayed (empty response or friendly error).
Verify that the input field becomes available again (does not remain in endless loading state).

Validation: Graceful behavior — no crash; UI returns to interactive state.

13.11 Toggle add_current_date_tool works `[ ]`

File: agent-current-date-tool.spec.ts

Objective: Verify that the Add Current Date Tool toggle adds/removes the date tool from the agent.

Precondition: Provider with valid API key configured.

Step by step:

Load "Simple Agent" template.
Enable the Add Current Date Tool toggle in the Agent component.
Open Playground and send: "What is today's date?".
Verify that the agent uses the date tool (appears in Agent Steps) and returns the correct date.
Disable the Add Current Date Tool toggle.
Send the same question.
Verify that the date tool does not appear in Agent Steps.

Validation: Toggle controls the presence of the date tool; tool appears/disappears in steps as configured.

13.12 handle_parsing_errors controls behavior on parse failure `[ ]`

File: agent-parse-error-behavior.spec.ts

Objective: Verify the difference in behavior between handle_parsing_errors=True and False.

Step by step (via mock):

Configure handle_parsing_errors = False in the Agent.
Intercept the LLM response to return malformed JSON when output_schema is configured.
Send a message.
Verify that the Agent returns an explicit error (does not try to correct).

Scenario B — True:

Configure handle_parsing_errors = True.
Repeat the same mock.
Verify that the Agent tries to self-correct (sends a second request) or returns a partial response.

Validation: Distinct behaviors according to handle_parsing_errors.

13.13 Image via input handle is processed by Agent `[ ]`

File: agent-multimodal-image-input.spec.ts

Objective: Verify that images passed via the input handle (not through the playground) are processed correctly by the agent.

Precondition: Multimodal model configured (e.g.: claude-3-5-sonnet, gpt-4o).

Step by step:

Add to canvas: Agent + component that generates an image (e.g.: URL Extractor with a public image).
Connect the image output to the Agent's input handle.
Configure Agent Instructions = "Describe what you see in the image.".
Execute the flow.
Verify that the Agent returns a description of the image (not an error or empty response).

Validation: Image content processed correctly via input handle.

14. Authentication — Login and Logout

Files: core/features/auto-login-off.spec.ts, login-invalid-credentials.spec.ts, logout-flow.spec.ts

14.1 Login with valid credentials `[-]`

Objective: Verify that a user with correct credentials accesses the system.

Precondition: Auto-login disabled (LANGFLOW_AUTO_LOGIN=false).

Step by step:

Navigate to http://localhost:7860.
Verify that the login screen is displayed.
Fill Username: langflow and Password: langflow.
Click "Sign In".
Verify that the user is redirected to the main page (mainpage_title visible).

Validation: User authenticated and redirected to the home.

14.2 Login with invalid credentials `[-]`

Step by step:

Navigate to the login screen.
Fill Username: wrong_user and Password: wrong_password.
Click "Sign In".
Verify that the error message "Error signing in" is displayed.
Verify that the user remains on the login screen.

Validation: Error message displayed, access blocked.

14.3 Logout redirects to login screen `[-]`

Step by step:

Login successfully.
Click the profile icon (user-profile-settings).
Click "Logout".
Verify that the user is redirected to the login screen.

Validation: Session ended and user redirected to login.

14.4 Auto-login enabled — skips login screen `[-]`

Step by step:

Navigate to http://localhost:7860 with LANGFLOW_AUTO_LOGIN=true.
Verify that the login screen is NOT displayed.
Verify that the main page loads directly.

Validation: With auto-login active, user accesses directly without credentials.

14.5 Auto-login disabled — displays login screen `[-]`

Step by step:

Mock the /api/v1/auto_login endpoint to return status 500.
Navigate to http://localhost:7860.
Verify that the login screen is displayed (text=sign in to langflow).

Validation: Without auto-login, the authentication screen is mandatory.

14.6 Expired session — redirects to login `[-]`

Step by step:

Login successfully.
Simulate token expiration (via mock or wait for timeout).
Attempt an authenticated action (e.g.: create flow).
Verify that the system redirects to the login screen.

Validation: Action with expired session results in redirect to login.

14.7 Session cleanup after logout `[-]`

Step by step:

Login and create a flow.
Logout.
Verify that session cookies/tokens were removed.
Attempt to access an authenticated URL directly — should redirect to login.

Validation: Session tokens cleared after logout.

15. User Management (Admin)

File: core/features/admin-user-management.spec.ts

15.1 Admin creates new user `[-]`

Step by step:

Login as admin.
Navigate to Admin Page (user menu → "Admin Page").
Click "New User".
Fill in the name, username and password of the new user.
Click save.
Verify success message "new user added".
Verify that the user appears in the listing.

Validation: New user created and visible in the listing.

15.2 Admin deactivates user `[-]`

Step by step:

Locate active user in Admin Page listing.
Click the #is_active toggle to deactivate.
Try to login with the deactivated user.
Verify that the login fails.

Validation: Deactivated user cannot authenticate.

15.3 Admin activates inactive user `[-]`

Step by step:

Locate inactive user.
Click the #is_active toggle to activate.
Login with the reactivated user.
Verify that the login is successful.

Validation: Activated user can authenticate normally.

15.4 Admin renames user `[-]`

Step by step:

Click the edit icon (icon-Pencil) of the user.
Change the display name.
Save.
Verify message "user edited".
Verify that the new name appears in the listing.

Validation: User name updated in the listing.

15.5 Admin changes user password `[-]`

Step by step:

Edit user and change password to "NewPassword123".
Save.
Try login with the old password — should fail.
Try login with the new password — should work.

Validation: Old password invalid, new password works.

15.6 Isolation: user A cannot see user B's flows `[-]`

Step by step:

Create flow with user A.
Login as user B.
Verify that user A's flows do NOT appear in user B's listing.

Validation: Flows are isolated per user.

16. Global Variables (API Keys)

Files: ui-ux/global-variable-edit.spec.ts, ui-ux/global-variables-crud.spec.ts

16.1 Create global variable from Settings page `[x]`

Objective: Confirm that a Generic global variable can be created from the Settings page (/settings/global-variables) and appears in the ag-grid table.

Step by step:

Navigate to Settings → Global Variables (/settings/global-variables).
Click the "Add New" button (api-key-button-store).
Switch to the Generic tab (generic-tab).
Fill in name and value.
Click Save (save-variable-btn).

Validation: The variable name appears as an exact match in .ag-cell-value within 10s.

16.2 Edit existing global variable `[x]`

Objective: Confirm that clicking an existing variable row opens the Update modal and saving a new value emits the "updated successfully" toast.

Step by step:

Create a variable as in 16.1.
Click the variable row in the ag-grid table.
Verify the "Update Variable" heading is visible.
Replace the value field with a new value.
Click Save (save-variable-btn).

Validation: Text matching /updated successfully/ is visible within 5s — the toast only fires when PATCH /api/v1/variables/{id} returns 200.

16.3 Delete global variable `[x]`

Objective: Confirm that deleting a variable removes it from the listing.

Step by step:

Locate a global variable in the listing.
Click the delete icon (icon-Trash2).
Confirm the deletion in the dialog.

Validation: The variable no longer appears in the listing (count drops to 0 for that name).

16.4 Create global variable of type "Generic" `[x]`

Objective: Confirm that the Generic tab is selectable and produces a Generic-type variable.

Step by step:

Open the Add New modal (either via the Globe icon in a component or via the Settings page).
Switch to the Generic tab.
Fill in name and value, save.

Validation: Generic type variable created with correct type, listed in the table.

16.5 Credential variable value is hidden from the variable list `[x]`

Objective: Confirm that after saving a Credential-type variable, its value is never rendered as visible text anywhere on the page (toast, label, preview, etc.).

Step by step:

Open the Add New modal.
Switch to the Credential tab.
Fill in name and a distinctive sentinel value (e.g. SECRET-SENTINEL-{Date.now()}).
Save.

Validation: getByText(sentinelValue) has count 0 — the sentinel must not surface as visible text anywhere in the DOM. Input value attributes (<input type="password" value="…">) don't count as visible text; only rendered text does, which is the guarantee under test.

17. File Upload and Processing

Files: core/unit/fileUploadComponent.spec.ts, extended/features/files-page.spec.ts

17.1 Upload file via component `[-]`

Step by step:

Add file upload component to canvas.
Click the upload button of the component.
Select file (e.g.: test.txt).
Verify that the file name appears in the component after upload.

Validation: File loaded and name displayed in the component.

17.2 Upload files of different types `[-]`

Step by step:

Test uploading .txt, .pdf, .json, .py files.
Verify that all types are accepted without error.

Validation: Multiple file formats accepted by the component.

17.3 File size limit `[-]`

Step by step:

Try to upload a file that exceeds the configured limit.
Verify that the system displays a size error message.
Verify that the file is NOT uploaded.

Validation: Error message displayed for file above the limit.

18. LLM Agents — Execution and Control

Files: llm-agents/agent-component-regression.spec.ts, llm-agents/memory-history-regression.spec.ts

agent-component-regression.spec.ts

18.1 Agent responds without connected tools `[x]`

Objective: Verify that the agent executes and returns a valid response even without any connected tool.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send: "What is the capital of France?".
Wait for execution to finish (Stop button disappears or never appears).
Verify that div-chat-message is visible.
Verify that the response text has content (length > 1).

Validation: Agent responds correctly without connected tools.

18.2 Agent displays reasoning steps and returns valid response `[x]`

Objective: Verify that the Agent responds with valid content and, when using internal reasoning, displays the duration indicator in the Playground. The steps check is soft — models that respond directly without tools do not generate the indicator, which is expected behavior.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send: "Who was the first astronaut to walk on the Moon?".
Wait for execution to finish (Stop button disappears or never appears — both valid).
Verify that div-chat-message is visible and has content (length > 1).
(Soft check) If "Finished in Xs" is visible, verify it is not empty.

Relevant DOM:

div-chat-message → assistant message
"Finished in" → duration indicator, displayed when the agent uses reasoning steps

Validation: Valid response returned for all models; duration indicator verified when present.

18.3 Agent stop button halts execution mid-run `[x]`

Objective: Verify that the Stop button interrupts agent execution during an ongoing run.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send: "Write a detailed story about the life and adventures of a fictional explorer in the 18th century.".
Wait for Stop button to appear (timeout 30s). If it doesn't appear, model responded too fast — implicit skip.
Click the Stop button via dispatchEvent("click").
Verify that the Stop button disappears.
Verify that input-chat-playground becomes visible again.

Validation: Execution interrupted successfully and playground returns to input state.

18.4 Agent displays duration after successful run `[x]`

Objective: Verify that the execution time is displayed at the end of a successful run.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send: "What are the main differences between mammals and reptiles?".
Wait for execution to finish (Stop button disappears or never appears).
Verify that div-chat-message is visible.
(Soft check) If "Finished in Xs" is visible, verify it is not empty.

Validation: Duration indicator displayed when present after successful run.

18.5 Agent streams response progressively in the playground `[x]`

Objective: Verify that the agent's response is displayed progressively in the playground, confirming that streaming is active during generation.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send: "Write a 5-paragraph summary explaining what artificial intelligence is, covering its definition, history, main techniques, applications, and future perspectives.".
Wait for div-chat-message to appear (agent started responding).
Capture the text at that moment (textAtStart).
Wait 3 seconds.
Capture the text again (textAfterWait).
If Stop button is still visible: assert that textAfterWait.length > textAtStart.length.
Wait for Stop to disappear and verify that the final text has content (length > 1).

Validation: Text grows progressively during streaming; final response with valid content.

18.6 Playground displays response time on canvas after closing `[x]`

Objective: Verify that after the agent finishes responding and the playground is closed, the duration indicator is displayed on the agent node in the canvas.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send: "What are the main differences between mammals and reptiles?".
Wait for the response to finish (Stop button disappears) and div-chat-message to be visible.
Click playground-close-button to close the playground.
Verify that node_duration_agent is visible on the canvas.

Relevant DOM:

playground-close-button → button to close the playground
node_duration_agent → duration indicator on the agent node in the canvas

Validation: Duration indicator displayed on canvas after closing the playground.

18.7 Agent handles multiple consecutive messages in same session `[x]`

Objective: Verify that the agent responds correctly to multiple sequential messages in the same session.

Step by step:

Load "Simple Agent" template and configure model via models.json.
Open Playground.
Send "Hello." and wait for response.
Send "Name three countries in South America." and wait for response.
Verify that there are at least 2 visible messages (div-chat-message count ≥ 2).

Validation: Agent responds correctly to both messages in the same session.

memory-history-regression.spec.ts

18.8 Memory History retains context between messages in the same session `[x]`

Objective: Verify that the Message History component maintains the conversation history between messages within the same Playground session.

File: llm-agents/memory-history-regression.spec.ts

Step by step:

Load "Memory Chatbot" template and configure OpenAI model.
Open Playground and start a new session (new-chat).
Send: "In our conversation my name is TESTNAME_XY9Z.".
Wait for response (1 message displayed).
Send: "What is my name from our conversation?".
Wait for response (2 messages displayed).
Verify that the response contains "TESTNAME_XY9Z".

Validation: Assistant recalls the name provided in the previous message.

18.9 Session isolation: distinct session IDs have independent histories `[x]`

Objective: Verify that two distinct sessions do not share history.

File: llm-agents/memory-history-regression.spec.ts

Step by step:

Load "Memory Chatbot" template and configure OpenAI model.
Session A: send "In our conversation my secret code is ALPHA_CODE_111.".
Start a new session (new-chat) — session B: send "What secret code did I mention?".
Verify that session B's response does not contain "ALPHA_CODE_111".

Validation: Session B has no access to session A's history.

18.10 Messages persist after closing and reopening the Playground `[x]`

File: llm-agents/memory-history-regression.spec.ts

Step by step:

Load "Memory Chatbot" template and configure OpenAI model.
Open Playground, start a new session and send: "In our conversation my value is PERSIST_VALUE_42.".
Close the Playground and reopen it by clicking playground-btn-flow-io.
Select the same session and send: "What value did I mention earlier?".
Verify that the response contains "PERSIST_VALUE_42".

Validation: History persisted between Playground openings.

18.11 Without Message History, LLM does not retain context between messages `[x]`

File: llm-agents/memory-history-regression.spec.ts

Step by step:

Load "Simple Agent" template (without Message History) and configure OpenAI model.
Open Playground, send: "In our conversation my secret is NOMEM5678.".
Send: "What secret did I just tell you?".
Verify that the response does not contain "NOMEM5678".

Validation: LLM without memory does not recall information from previous messages.

18.12 n_messages parameter of Message History `[ ]`

File: agent-n-messages-limit.spec.ts — awaiting backend bug fix.

⚠️ Confirmed bug: The n_messages parameter is saved correctly by the frontend but ignored during backend execution (MemoryComponent.retrieve_messages()).

Step by step (when the bug is fixed):

Load "Memory Chatbot" template, change n_messages to 2 in the "Message History" node.
Send 3 exchanges with distinct values (ALPHA, BETA, GAMMA).
Ask for all 3 values.
Verify that GAMMA is in the response and ALPHA is not.

Validation: With n_messages=2, only the last 2 message pairs are in context.

18.13 Fixed context_id — continuity between messages `[ ]`

File: agent-context-id-continuity.spec.ts

Objective: Verify that the Agent maintains memory between messages when a fixed context_id is configured.

Precondition: Provider with valid API key. Agent with context_id defined (e.g.: "test-session-001").

Step by step:

Load "Simple Agent" template.
Configure context_id = "test-session-001" in the Agent component.
Open Playground.
Send: "My name is John". Wait for response.
Send: "What is my name?". Wait for response.
Verify that the response contains "John".

Validation: Agent recalls information from the previous message via context_id.

18.14 Switch context_id — isolation between sessions `[ ]`

File: agent-context-id-isolation.spec.ts

Objective: Verify that changing the context_id starts a new context, without access to the previous history.

Precondition: Provider with valid API key.

Step by step:

Configure context_id = "session-A" in the Agent.
Send: "My name is Ana". Wait for response.
Change context_id to "session-B".
Send: "What is my name?".
Verify that the response does not mention "Ana" (isolated session).

Validation: History of "session-A" does not leak into "session-B".

18.15 Multiple tools — agent selects the correct one `[ ]`

File: agent-multi-tool-selection.spec.ts

Objective: Verify that the agent chooses the correct tool among multiple available ones.

Precondition: Two tools connected to the Agent (e.g.: Calculator + DuckDuckGo).

Step by step:

Connect Calculator and DuckDuckGo to the Agent.
Open Playground.
Send: "What is 47 times 83?".
Wait for response.
Verify in "Agent Steps" that Calculator was called (not DuckDuckGo).
Send: "Search for the latest news about artificial intelligence".
Verify in "Agent Steps" that DuckDuckGo was called (not Calculator).

Validation: Agent selects the correct tool based on the nature of the prompt.

18.16 Tool that returns error — agent does not crash `[ ]`

File: agent-tool-error-handling.spec.ts

Objective: Verify agent behavior when a connected tool returns an error.

Step by step:

Create a custom component that always raises an exception (e.g.: raise ValueError("tool error")).
Connect as a tool to the Agent.
Send a prompt that forces use of the tool.
Wait for execution.
Verify that:
- The Playground does not freeze indefinitely.
- The tool error appears in "Agent Steps" (ToolContent with error status).
- The agent returns some response (alternative or friendly error message).

Validation: Tool failure is handled; agent does not crash; error visible in steps.

18.17 Tool with invalid name — validation prevents execution `[ ]`

File: agent-tool-name-validation.spec.ts

Objective: Verify that the Agent validates tool names and rejects names outside the pattern ^[a-zA-Z0-9_-]+$.

Step by step:

Create a custom component with a name containing a space or special character (e.g.: "my tool!").
Connect as a tool to the Agent.
Execute the flow.
Verify that the Agent component displays a validation error before calling the LLM.
Verify that the error message is clear and visible on the canvas (not a silent error).

Validation: Tool name validation occurs before execution; explicit error displayed.

19. Model Providers

Files: claude-model-switch.spec.ts, modelProviderModal.spec.ts, provider-invalid-auth-error.spec.ts

19.1 Configure OpenAI API key via Global Variables `[-]`

Step by step:

Navigate to Settings → Global Variables.
Create variable OPENAI_API_KEY with the valid key.
Add OpenAI component to canvas.
Verify that the API key field displays the global variable as an option.
Select the global variable.

Validation: API key configured via global variable.

19.2 Select GPT model (GPT-4o-mini) `[-]`

Step by step:

Add OpenAI component to canvas.
Click the model dropdown.
Select gpt-4o-mini (testid: gpt-4o-mini-option).
Verify that the selected model appears in the dropdown.

Validation: GPT-4o-mini model selected and displayed.

19.3 Select Claude model `[-]`

Step by step:

Add Anthropic component to canvas.
Configure Anthropic API key.
Select Claude model (e.g.: claude-sonnet-4-5-20250929).
Verify that the model is selected.

Validation: Claude model selected correctly.

19.4 Switch between Claude models `[-]`

Step by step:

With Anthropic component configured with Sonnet model.
Open dropdown and select Haiku.
Verify that the model changes.
Repeat for Opus.

Validation: All Claude models available and selectable.

19.5 Invalid authentication error — OpenAI `[-]`

File: provider-invalid-auth-error.spec.ts

Precondition: OPENAI_API_KEY configured in .env.

Step by step:

Navigate to Settings via user menu.
Click the icon-Brain icon to access Model Providers.
Click the OpenAI provider.
Select all content in the API key field and enter an invalid key (e.g.: sk-invalid-openai-key-for-testing).
Click Save Configuration (first time) or Replace Configuration (key already exists).
Verify that .error-build-message displays text matching /Invalid API key/i.
(Cleanup) Select the field, enter the valid key from .env and click Replace Configuration.

Validation: Langflow displays .error-build-message with /Invalid API key/i immediately after saving an invalid key on the provider configuration page.

19.6 Invalid authentication error — Anthropic `[-]`

File: provider-invalid-auth-error.spec.ts

Precondition: ANTHROPIC_API_KEY configured in .env.

Step by step:

Navigate to Settings via user menu.
Click the icon-Brain icon to access Model Providers.
Click the Anthropic provider.
Select all content in the API key field and enter an invalid key (e.g.: sk-ant-invalid-for-testing).
Click Save Configuration or Replace Configuration.
Verify that .error-build-message displays text matching /Invalid API key/i.
(Cleanup) Select the field, enter the valid key from .env and click Replace Configuration.

Validation: Langflow displays .error-build-message with /Invalid API key/i immediately after saving an invalid key on the provider configuration page.

19.7 Invalid authentication error — Google `[-]`

File: provider-invalid-auth-error.spec.ts

Precondition: GOOGLE_API_KEY configured in .env.

Step by step:

Navigate to Settings via user menu.
Click the icon-Brain icon to access Model Providers.
Click the Google Generative AI provider.
Select all content in the API key field and enter an invalid key (e.g.: AIza-invalid-google-key-for-testing).
Click Save Configuration or Replace Configuration.
Verify that .error-build-message displays text matching /Invalid API key/i.
(Cleanup) Select the field, enter the valid key from .env and click Replace Configuration.

Validation: Langflow displays .error-build-message with /Invalid API key/i immediately after saving an invalid key on the provider configuration page.

19.8 "Manage Model Providers" modal `[-]`

Step by step:

Click the provider management button.
Verify that the "Manage Model Providers" modal opens.
Verify the list of available providers.
Click on a provider and verify that it is possible to configure the API key.

Validation: Modal opens, lists providers and allows configuration.

19.9 Configure provider API key — first setup (Save Configuration) `[x]`

File: helpers/provider-setup/collect-models.ts — collectModelsForProvider

Objective: Verify that the helper can configure an API key on a provider that does not yet have a saved key.

Precondition: Provider without API key configured in Langflow (field with placeholder sk-ant-..., AIza... or sk-... visible).

Step by step:

Navigate to Settings → Model Providers.
Click the desired provider (e.g.: Anthropic).
Verify that the input with placeholder sk-ant-... is visible.
Click the input — "Save Configuration" button appears enabled.
Type the API key via pressSequentially.
Click "Save Configuration".
Wait for the "Replace Configuration" button to appear on screen as confirmation.

Validation: "Replace Configuration" button displayed after save, indicating the key was persisted.

19.10 Replace provider API key — existing key (Replace Configuration) `[x]`

File: helpers/provider-setup/collect-models.ts — collectModelsForProvider

Objective: Verify that the helper can replace an API key already configured on a provider.

Precondition: Provider with API key already saved ("Replace Configuration" button present).

Step by step:

Navigate to Settings → Model Providers.
Click the desired provider.
Verify that the input with placeholder sk-ant-... is visible.
Click the input — previous value disappears and "Replace Configuration" button becomes disabled.
Type the new API key via pressSequentially — the React onChange enables the button.
Click "Replace Configuration".
Wait for the "Replace Configuration" button to reappear as confirmation.

Validation: "Replace Configuration" button redisplayed after click, confirming the new key was saved.

20. Observability — Traces and Notifications

Files: core/features/traces.spec.ts, traces-latency-tokens.spec.ts, execution-error-notification.spec.ts

20.1 View execution traces `[-]`

Step by step:

Execute a flow at least once.
Navigate to the Traces/Logs section.
Verify that the execution appears in the traces list.
Click on the trace to expand details.

Validation: Execution trace visible with expandable details.

20.2 Trace displays latency of each component `[-]`

Step by step:

Access the detail of a trace.
Verify that each flow component displays its latency (execution time).

Validation: Per-component latency visible in trace details.

20.3 Trace displays tokens consumed `[-]`

Step by step:

Execute flow with LLM.
Access the execution trace.
Verify that token fields (input tokens, output tokens, total) are present.

Validation: Token count visible in the trace.

20.4 Execution error notification `[-]`

Step by step:

Create flow with a component that will generate an error (e.g.: invalid URL in API Request).
Execute the flow.
Verify that an error notification appears in the interface with a descriptive message.

Validation: Execution error displayed as notification with details.

21. Playground — Chat and Session

Files: core/features/playground-ux.spec.ts, playground-session-id.spec.ts, playground-history-persist.spec.ts

21.1 Open Playground `[-]`

Step by step:

With a flow created and open in the editor.
Click the Playground button (playground-btn-flow-io).
Verify that the Playground panel opens.
Verify that the input field (input-chat-playground) is visible.

Validation: Playground opens with chat interface ready to use.

21.2 Send message and receive response `[-]`

Step by step:

Open Playground.
Type message in the input field.
Click "Send" (button-send).
Wait for response.
Verify that the assistant's response appears in the history.

Validation: Message sent, response received and displayed.

21.3 Send empty message `[!]` (DOCUMENTED BUG)

⚠️ KNOWN BUG: The send button is always enabled even with an empty field.

Step by step:

Open Playground without typing anything.
Check the state of the "Send" button.
Click "Send" with empty field.

Validation: Documented as bug — button should be disabled with empty field.

21.4 Switch session ID — starts new conversation `[-]`

Step by step:

Send message in the current session.
Locate the chat-session-id field and type a new ID.
Verify that the chat history is cleared (new session).
Confirm it is an independent conversation.

Validation: New session created without previous session's history.

21.5 Delete individual message from history `[-]`

Step by step:

Send at least one message in the Playground.
Hover over the message to show options.
Click the delete icon of the message.
Verify that the message was removed from the history.

Validation: Deleted message no longer appears in history.

21.6 History persists when reopening Playground `[-]`

Step by step:

Send messages in the Playground.
Close the Playground panel.
Reopen the Playground.
Verify that the previous messages are still in the history.

Validation: Chat history preserved after closing and reopening.

21.7 Playground fullscreen mode `[-]`

Step by step:

Open Playground.
Click the fullscreen button.
Verify that the Playground occupies the full screen.
Verify that it is still possible to send messages.

Validation: Fullscreen active, chat functionality preserved.

21.8 Shareable Playground — URL generation `[-]`

File: tests/tests-automations/regression/core-functionality/playground/playground-shareable-url.spec.ts

Objective: Verify that enabling the Shareable Playground feature on a flow with Chat I/O generates a valid public URL in the format /playground/{uuid}.

Preconditions: Langflow running. The ENABLE_PUBLISH feature flag must be active (enabled by default). Flow must contain Chat Input and Chat Output (Simple Agent template satisfies this).

Step by step:

Load the Simple Agent template.
Click the publish-button (Share button in the flow toolbar) to open the Share dropdown.
Verify that the shareable-playground item is visible and the publish-switch is unchecked (sharing off by default).
Click publish-switch to enable sharing.
Verify the switch becomes checked.
Verify that a link <a href="/playground/{uuid}"> appears inside the shareable-playground item.
Assert the href matches /\/playground\/[0-9a-f-]{36}/.
Click publish-switch again to disable sharing (cleanup).

Validation: After enabling the switch, [data-testid="shareable-playground"] a is visible and its href attribute matches the /playground/{uuid} pattern. The switch returns to unchecked after cleanup.

22. Project and Folder Management

File: core/features/folders.spec.ts, folder-deletion-integrity.spec.ts

22.1 Create new folder `[-]`

Step by step:

On the main page, click "New Folder".
Type the folder name.
Confirm creation.
Verify that the folder appears in the listing.

Validation: Folder created and visible in the projects sidebar.

22.2 Rename folder `[-]`

Step by step:

Click the folder edit icon.
Type new name and confirm.
Verify that the new name appears in the listing.

Validation: Folder name updated.

22.3 Delete empty folder `[-]`

Step by step:

Create empty folder.
Click delete and confirm.
Verify that the folder no longer appears in the listing.

Validation: Empty folder deleted successfully.

22.4 Delete folder with flows inside `[-]`

Step by step:

Create folder with at least one flow inside.
Try to delete the folder.
Confirm deletion (cascade or alert).
Verify that folder and flows were removed.

Validation: Cascade deletion works or alert is displayed.

22.5 Move flow to another folder `[-]`

Step by step:

Select a flow and use "Move to Folder" option.
Select the target folder.
Verify that the flow appears in the new folder and not in the original.

Validation: Flow moved to target folder correctly.

22.6 Search flow by name `[-]`

Step by step:

Create at least two flows with different names.
Use the search field on the main page.
Type part of a flow name.
Verify that only the matching flow is displayed.

Validation: Search filters flows by name correctly.

23. Templates and Starter Projects

Files: core/integrations/*.spec.ts

23.1 Basic Prompting (OpenAI) `[-]`

Step by step:

Select "Basic Prompting" template.
Verify that the flow loads with Chat Input, Prompt Template, OpenAI LLM, Chat Output.
Configure OpenAI API key.
Open Playground and send a message.
Verify that a response is received.

Validation: Basic Prompting template executes and returns OpenAI response.

23.2 Simple Agent (OpenAI) `[-]`

Step by step:

Select "Simple Agent" template.
Verify that flow loads with Agent and default tools.
Configure API key.
Send a question in the Playground.
Verify Agent response.

Validation: Agent executes and returns response via Simple Agent template.

23.3 Memory Chatbot `[x]`

Objective: Verify that the Memory Chatbot template loads correctly and that the chatbot maintains context between messages.

File: llm-agents/memory-history-regression.spec.ts

Step by step:

Navigate to "All Templates" and select "Memory Chatbot".
Wait for canvas to load (canvas_controls_dropdown visible).
Verify that there are at least 3 nodes on the canvas (Memory History, LLM, Chat I/O).
Verify that there are at least 2 edges (connections between nodes).
Verify that the Playground button is visible (playground-btn-flow-io).
Configure OpenAI API key, open Playground and start new session.
Send: "In our conversation my name is TESTNAME_XY9Z.".
Send: "What is my name from our conversation?".
Verify that the response contains "TESTNAME_XY9Z".

Validation: Template loads with correct structure; chatbot maintains conversation context between messages.

23.4 Vector Store RAG `[-]`

Step by step:

Load "Vector Store RAG" template.
Upload a test document.
Configure embeddings and vector store.
Send a question related to the document content.
Verify that the response is based on the document.

Validation: RAG returns response based on the loaded document.

24. Flow — CRUD and Operations

Files: core/features/export-import-flow.spec.ts, flow-lock.spec.ts, run-flow.spec.ts

24.1 Create blank flow `[-]`

Step by step:

Click "New Flow" and select "Blank Flow".
Verify that an empty canvas is displayed.

Validation: Empty canvas displayed after selecting Blank Flow.

24.2 Create flow by duplicating existing `[-]`

Step by step:

Click "Duplicate" in a flow menu.
Verify that a new flow with "(copy)" in the name is created with the same components.

Validation: Flow duplicated with a copy of the original components.

24.3 Import flow via JSON `[-]`

Step by step:

Click "Import" and select a flow JSON file.
Verify that the flow is imported and displayed in the editor.

Validation: Flow imported correctly from JSON file.

24.4 Export flow as JSON `[-]`

Step by step:

Open existing flow.
Click Export.
Verify that a .json file download starts with the correct structure.

Validation: Valid JSON file generated for the flow.

24.5 Import invalid JSON — shows error `[-]`

Step by step:

Try to import a .json file with invalid content.
Verify that an error message is displayed.
Verify that no invalid flow is created.

Validation: Invalid JSON import displays descriptive error.

24.6 Lock flow `[-]`

Step by step:

Click the lock button in the editor.
Try to move a component on the canvas.
Verify that the movement is prevented.

Validation: Locked flow prevents edits on the canvas.

24.7 Execute flow via Run button `[-]`

Step by step:

Click the "Run" button (button_run_flow).
Verify that execution starts (loading indicators on components).
Verify that outputs are displayed at the end.

Validation: Execution started and results displayed in components.

24.8 Stop building flow `[-]`

Step by step:

Start flow execution.
Click "Stop" (stop-building-button) during execution.
Verify that execution stops and the "Run" button becomes available again.

Validation: Execution interrupted when clicking Stop.

25. MCP — Client and Server

25.1 MCP Server tab in flow `[-]`

Step by step:

Open a flow in the editor.
Verify the presence of the "MCP Server" tab.
Click the tab and verify that MCP-related content is displayed.

Validation: MCP Server tab accessible in the flow editor.

25.2 Add MCP server via modal `[-]`

Step by step:

Navigate to MCP configuration.
Click "Add MCP Server".
Fill in configuration (name, URL/command) and save.
Verify that the MCP server appears in the listing.

Validation: MCP server added and visible in the listing.

25.3 Configure MCP client connection `[ ]`

Step by step:

Add MCP Client component to the flow.
Configure connection type (stdio or HTTP) and parameters.
Verify that the connection is established without errors.

Validation: MCP Client component connected successfully.

26. UI/UX — Sidebar and Canvas

26.1 Search component by name `[-]`

Step by step:

Type name in the sidebar-search-input field.
Verify that only matching components are displayed.
Clear with .clear().
Verify that all components return.

Validation: Search filter works and clears correctly.

26.2 Drag component from sidebar to canvas `[-]`

Step by step:

Locate component in the sidebar.
Drag to a specific position on the canvas.
Verify that the component appears on the canvas.

Validation: Component added to canvas via drag-and-drop.

26.3 Double-click in sidebar adds component `[-]`

Step by step:

Locate component in the sidebar.
Double-click on the component.
Verify that the component is added to the canvas automatically.

Validation: Double-click adds component to canvas.

26.4 Connect two compatible components `[-]`

Step by step:

Add Chat Input and Chat Output to canvas.
Click on the output handle of Chat Input.
Click on the input handle of Chat Output.
Verify that an edge is created.

Validation: Edge visible connecting the two components.

26.5 Prevent connection between incompatible types `[-]`

Step by step:

Try to connect handles of incompatible types.
Verify that the connection is not allowed.

Validation: System prevents connection between handles of incompatible types.

26.6 Delete component from canvas `[-]`

Step by step:

Select component on canvas.
Press Delete or use context menu → Delete.
Verify that the component is removed.

Validation: Component removed after Delete key.

26.7 Copy and paste component (Ctrl+C / Ctrl+V) `[-]`

Step by step:

Click on component to select it.
Press Ctrl+C.
Click on empty area of canvas.
Press Ctrl+V.
Verify that a second component (copy) appears.

Validation: Canvas with 2 components after copy-paste.

26.8 Select multiple components via box selection `[-]`

Step by step:

Add 2+ components to canvas.
Hold Shift and drag to create a selection box covering the components.
Verify that all covered components become selected.

Validation: Multiple components selected via Shift+drag.

26.9 Minimize component on canvas `[-]`

Step by step:

Click the minimize button of the component.
Verify that the component displays the minimized version.
Click again to expand.
Verify that the component returns to normal size.

Validation: Component minimizes and expands correctly.

26.10 Zoom in / Zoom out / Fit View `[-]`

Step by step:

Click Zoom In — verify that scale increases.
Click Zoom Out — verify that scale decreases.
Press Ctrl+Shift+H or click "Fit View" — verify that canvas centers all nodes.

Validation: Zoom and fit view work as expected.

26.11 Create and undo grouping `[-]`

Step by step:

Select 2+ components via box selection.
Use group option (context menu → "Group").
Verify that a group component is created.
Use "Ungroup" and verify that the original components are restored.

Validation: Component grouping and ungrouping works.

26.12 Freeze component `[-]`

Step by step:

Click "Freeze" on the component.
Verify visual indication of frozen.
Execute flow — verify that the frozen component uses cache.

Validation: Frozen component does not re-execute on new flow run.

26.13 Add and delete sticky note `[-]`

Step by step:

Right-click on canvas → "Add Note".
Verify that sticky note appears on canvas.
Select and press Delete.
Verify that the note was removed.

Validation: Sticky note added and removed from canvas.

26.14 Change sticky note color `[-]`

Step by step:

Select sticky note on canvas.
Choose a different color in the color selector.
Verify that the sticky note color changed.

Validation: Sticky note color changed as selected.

26.15 Context menu via right-click on canvas `[-]`

Step by step:

Right-click on empty area of canvas.
Verify that context menu opens with available options.

Validation: Context menu opens with correct options.

26.16 Access Settings page `[-]`

Step by step:

Click the profile icon (user-profile-settings).
Click "Settings" (menu_settings_button).
Verify that the Settings page loads with all tabs.

Validation: Settings page accessible with all tabs.

26.17 Change appearance/theme settings `[-]`

Step by step:

Access Settings.
Locate the theme toggle (Dark/Light mode).
Click to toggle the theme.
Verify that the theme changes in the interface.

Validation: Interface theme changes as configured.

26.18 Edit sticky note text `[x]`

File: tests/tests-automations/regression/ui-ux/edit-sticky-note-text.spec.ts

Objective: Verify that editing an existing sticky note replaces its text — the canvas renders only the new content, not a mix of old and new. This is a distinct journey from adding a note (26.13), which only fills an empty note.

Preconditions: Langflow running.

Step by step:

Create a blank flow (via API) and zoom out so the note fits the viewport.
Click canvas-add-note-button to add a sticky note.
Double-click generic-node-desc to open the editor, fill "Original note content", and commit (click rf__wrapper + Escape).
Confirm the rendered note shows "Original note content".
Double-click the note again — confirm the editor pre-loads the existing text.
Clear and replace the text with "Edited note content", then commit again.
Read the rendered note text.

Validation: Rendered note contains "Edited note content" and does NOT contain "Original note content" — proving the edit replaced the content rather than appending to it.

Current Coverage Summary

Module	Total	Covered	Pending
REST API	17	17	0
Authentication + Users	17	15	2
Component Configuration	20	18	2
Core Components	22	16	6
Playground	17	14	3
Observability	16	13	3
Model Providers	19	10	9
Knowledge Ingestion	8	4	4
Flow Operations	20	18	2
MCP	13	3	10
Project Management	11	9	2
Templates	35	33	2
UI/UX Canvas	35	33	2
TOTAL	250	203 (81%)	47 (19%)

Automation Priorities

🔴 High Priority (release blockers)

Invalid API key error (OpenAI/Anthropic) — user must be clearly informed
Flow with Python error displays clear message in UI
Component update with breaking change — user alert
Network error during execution — retry or descriptive message

🟡 Medium Priority

MCP client — consumption of external tools and resources
Webhook trigger via external HTTP request
Agent — inspect tools used in Playground
[-] Shareable Playground URL generation (see 21.8)
Complete RAG pipeline

🟢 Low Priority

Loop component — correct iterations (covered in section 27)
Ollama, Groq, Mistral providers
Model parameters (temperature, max tokens)
[-] Edit sticky note text (covered in section 26.18; pending PR approval)
Use global variable directly in component

27. Core Components — Loop

File: tests/tests-automations/regression/core-components/loop-component-regression.spec.ts

27.1 Loop component renders on canvas with all handles `[x]`

Objective: Verify that the Loop appears correctly on the canvas with all input/output handles and output inspection buttons.

Preconditions: Langflow running.

Step by step:

Create a blank flow.
Search "Loop" in the sidebar and add to canvas.
Verify that the title title-Loop is visible.
Verify that the run button button_run_loop is visible.
Verify that there is exactly 1 node on the canvas.
Verify input handles (left side):
- handle-loopcomponent-shownode-inputs-left — receives the DataFrame to iterate
- handle-loopcomponent-shownode-item-left — feedback port: receives the processed item
Verify output handles (right side):
- handle-loopcomponent-shownode-item-right — emits the current iteration item
- handle-loopcomponent-shownode-done-right — emits the aggregated DataFrame at the end
Verify output inspection buttons in the node footer:
- output-inspection-item-loopcomponent
- output-inspection-done-loopcomponent

Validation: All handles and inspection buttons visible; canvas with exactly 1 node.

27.2 Loop without connections shows "Flow build failed" notification `[x]`

Objective: Verify that executing the Loop in isolation (without connections) results in a controlled failure — without application crash.

Preconditions: Langflow running.

Step by step:

Create a blank flow.
Add the Loop component to canvas.
Click button_run_loop (node with no connections).
Wait for the error notification to appear.

Validation:

Text "Flow build failed" visible on screen.
Button button_run_loop still accessible after failure.
Node title-Loop remains intact on canvas (1 node).
Application does not freeze or reload.

27.3 Loop iterates over 2 ArXiv articles via Research Translation Loop template `[x]`

Objective: Validate the complete wiring of the Loop and that it iterates correctly — each DataFrame item enters the cycle, is processed by the LLM and returns via the item port, until done fires with the aggregated result.

Preconditions: Langflow running, OPENAI_API_KEY configured.

Step by step:

Access the Templates tab (side_nav_options_all-templates).
Click the template template-research-translation-loop.
Wait for canvas to load with the Loop node visible.
Verify wiring:
- At least 1 edge present on canvas (template already connects the Loop in a cycle).
- Handles inputs-left, item-left, item-right and done-right of the Loop visible.
Reduce the int_int_max_results field of ArXiv to 2 (minimum to validate iteration without long wait).
Open the Playground (playground-btn-flow-io).
Send the message "transformer neural networks".
Wait for the bot response (chat-message-AI-).
Verify that the response contains at least 2 occurrences of "title" (case-insensitive) — each ArXiv article has a title, confirming the loop processed the 2 articles.

Validation: Non-empty bot response; "title" count ≥ 2 (confirms 2 loop iterations).

Note: Validation via "Title" is intentional and slightly fragile — it depends on the Parser's output format. If the template changes the prompt template, the counter may not match. The test's focus is to confirm the Loop iterated, not the exact content.

28. API Keys — Timestamps & Expiry

Files: ui-ux/api-keys-timezone-display.spec.ts, api/flows/api-key-expiry-enforcement.spec.ts Reference: PR #13471 — Fix timestamp rendering for expires_at in API Key model.

28.1 Serializer emits UTC offset, no microseconds `[x]`

Objective: Confirm GET /api/v1/api_key/ serializes datetime fields as offset-aware UTC ISO (+00:00) at second precision, the root-cause fix for the UTC display bug.

Precondition: Langflow running; authenticated (auto_login or form login).

Step by step:

Create two keys via POST /api/v1/api_key/: one with expires_at = 2026-06-10T23:59:59+00:00, one with no expiry.
GET /api/v1/api_key/ and locate both keys.
Verify created_at (both) and expires_at (expiring key) match ^…T…\+00:00$ with no microseconds.
Verify expires_at round-trips to 2026-06-10T23:59:59+00:00.
Verify no-expiry expires_at and both last_used_at are null.

Validation: All datetime fields offset-aware and second-precision; nulls preserved.

28.2 Settings table renders timestamps in local timezone `[x]`

Objective: Confirm the Settings → API Keys table converts UTC instants to the viewer's local time, with correct empty-state glyphs.

Precondition: Browser timezone pinned to America/Sao_Paulo (UTC−03:00); the two keys from 28.1 present.

Step by step:

Log in and open /settings/api-keys.
Verify the expiring key's expires cell reads 2026-06-10 20:59:59 (23:59:59 UTC − 03:00).
Verify the created cell is well-formatted and differs from the raw UTC wall clock.
Verify the unused key's last used cell reads Never and the no-expiry key's expires cell reads ∞.

Validation: expires_at shows 20:59:59 (not the pre-fix 23:59:59); Never and ∞ render correctly.

28.3 Expired key rejected, valid key accepted `[x]`

Objective: Confirm API key expiry is enforced on POST /api/v1/run/{id} (x-api-key).

Precondition: Langflow running; an empty flow created to run against.

Step by step:

Create an expired key (expires_at = 2020-01-01T00:00:00+00:00) and a valid key (2099-12-31T23:59:59+00:00).
Run the flow with the expired key → expect 403.
Run the flow with the valid key → expect 200.

Validation: Expired → 403 "Invalid or missing API key"; valid → 200.

28.4 Expiry boundary evaluated in UTC `[x]`

Objective: Confirm the expiry comparison is UTC-based and not shifted by the viewer's timezone offset.

Precondition: Langflow running; empty flow available.

Step by step:

Create a key expiring now + 30 min (UTC) and a key expiring now − 30 min (UTC).
Run with the near-future key → expect 200.
Run with the recently-expired key → expect 403.

Validation: Both verdicts correct — the 30-min margins sit inside the ±3h offset window, so a timezone-shifted comparison would flip one of them.

29. UI/UX — Settings Shortcuts

File: ui-ux/settings-shortcuts-edit.spec.ts

29.1 Editing a keyboard shortcut persists and triggers the action on canvas `[x]`

Objective: Confirm that editing a keyboard shortcut from Settings → Shortcuts both persists to the table and rebinds the action on the flow canvas — the store (useShortcutsStore → localStorage["langflow-shortcuts"]) and the canvas keybind handler must stay in sync.

Precondition: Langflow running and accessible.

Step by step:

Open Settings → Shortcuts.
Double-click the Duplicate row to open the edit modal.
Record Ctrl/Cmd+Alt+U and click Apply.
Confirm the toast "Duplicate shortcut successfully changed" and that the Duplicate row now shows the new combination.
Open a blank flow, add an Ollama node, select it, and press Ctrl/Cmd+Alt+U.
Confirm the node is duplicated (count 1 → 2).

Validation: The edited combination persists in the table and triggers Duplicate on the canvas (title-Ollama count goes from 1 to 2). afterEach restores defaults via the Restore button plus a localStorage safeguard.

30. Core Components — Nested / Grouping

File: tests/tests-automations/regression/core-components/nested-grouping-regression.spec.ts

30.1 Group two connected non-IO components collapses them into a single Group node `[x]`

Objective: Verify that box-selecting two connected non-IO components and clicking Group replaces them with a single title-Group node on the outer canvas — proving the components were nested inside a reusable subflow rather than renamed or duplicated.

Preconditions:

Langflow running.
No API key required (no LLM execution).
The test creates a custom 2-node flow via POST /api/v1/flows/ (Prompt Template → Language Model, no IO, no sticky notes) because the Group button gates on validateSelection from reactflowUtils.ts, which rejects IO nodes and sticky-note overlaps — both present in starter templates.

Step by step:

Create the flow via REST API using tests/assets/flows/two-non-io-connected.json and navigate to it via the home dashboard (avoids the /flow/{id} cache race after API creation).
Wait for canvas_controls_dropdown, title-Prompt Template and title-Language Model to be visible; call adjustScreenView.
Click the empty React Flow pane and wait for .react-flow__node.selected count to drop to 0.
Shift+drag a box covering both nodes' bounding boxes (with 40px padding).
Wait for .react-flow__node.selected count to reach 2 and getByTestId("group-node") to be visible.
Click the Group button (forced click — lives inside @xyflow/react's NodeToolbar portal).
Wait for the Group button to disappear (confirms the mutation committed).

Validation:

.react-flow__node count is exactly 1.
title-Group is visible.
title-Prompt Template and title-Language Model have count 0 on the outer canvas.

30.2 Ungrouping a Group node restores the original components and the edge `[x]`

Objective: Verify that right-clicking a Group node and triggering Ungroup re-emits the encapsulated subflow back to the outer canvas — restoring both original components and the edge that connected them, with no data loss.

Preconditions:

Same setup as 30.1.
The Group node must already exist on the canvas (created via 30.1's flow).

Step by step:

Repeat steps 1–7 of scenario 30.1 to produce a title-Group node on the canvas.
Right-click the title-Group element to open the node toolbar dropdown.
Click group-button-modal (the Ungroup entry — only rendered for Group-typed nodes per isGroup && <SelectItem value="ungroup"> in nodeToolbarComponent/index.tsx).

Validation:

.react-flow__node count is back to 2.
.react-flow__edge count is 1 (the original connection was preserved).
title-Group has count 0.
title-Prompt Template and title-Language Model are both visible again.

Note: Modern Langflow does not expose a separate nested-canvas view — "enter/exit grouped component" is the Group/Ungroup round-trip. This scenario validates the data fidelity of the round-trip.

Generated on 2026-03-18 | Source: QA-CHECKLIST.md

FilesExpand file tree

QA-SCENARIOS-GUIDE.md

Latest commit

History

QA-SCENARIOS-GUIDE.md

File metadata and controls

Langflow — Test Scenario Guide (Step by Step)

Index

1. REST API — Health Check

1.1.a GET /health_check → status 200 [-]

1.1.b GET /api/v1/version → returns version, main_version, package [x]

2. REST API — Flow CRUD

2.1 POST /api/v1/flows/ → creates flow, returns ID [-]

2.2 GET /api/v1/flows/ → lists user flows [-]

2.3 GET /api/v1/flows/{id} → returns flow by ID [-]

2.4 PATCH /api/v1/flows/{id} → updates name/description [-]

2.5 DELETE /api/v1/flows/{id} → removes flow, returns 200 [-]

2.6 GET after DELETE → should return 404 [-]

3. REST API — Flow Execution

3.1 POST /api/v1/run/{flow_id} with input_value [-]

3.2 POST with tweaks → parameters override configuration [-]

3.3 POST with custom session_id [-]

3.4 POST with invalid API key → returns 401/403 [-]

3.5 POST to non-existent flow → returns 404 [-]

4. REST API — Components and Messages

4.1 GET /api/v1/all → lists available components [-]

4.2 GET /api/v1/monitor/messages → returns array [-]

4.3 GET with session_id filter [-]

5. REST API — Integration Code Generation

5.1 Generate curl for execution [x]

5.2 Generate Python code for integration [x]

6. Component Configuration — Parameters Panel

6.1 Open component advanced options [-]

6.2 Edit text field (input) [-]

6.3 Edit dropdown [-]

6.4 Edit toggle [-]

6.5 Edit float / int field [-]

6.6 Edit slider [-]

7. Tool Mode

7.1 Enable Tool Mode on a component [-]

7.2 Group components in Tool Mode [-]

8. Component Updates

8.1 Outdated component notification [-]

8.2 Update component action [-]

9. Core Components — Chat Input/Output

9.1 ChatInput receives user message [-]

9.2 ChatOutput displays LLM response [-]

10. Core Components — Prompt Template

10.1 Prompt with variables in curly braces [x]

10.2 Removing variable from prompt deletes corresponding port [x]

11. Core Components — API Request

11.1 Renders on canvas with URL and API Response handles [x]

11.2 Inspector accepts URL and HTTP method values [x]

11.3 Invalid URL is accepted by field but run shows error notification [x]

11.4–11.8 Execute GET / POST / PUT / PATCH / DELETE returns 200 [x]

11.9 Non-2xx HTTP response (404) propagates as status_code without crashing [x]

11.10 Query parameters embedded in URL are sent and echoed [x]

11.11 Headers table accepts key + value cell entries via inspector [x]

11.12 cURL tab switches mode and field accepts a cURL command [x]

11.13 cURL parser auto-fills URL field and executes the GET, returning 200 [x]

12. Core Components — Webhook

12.1 Webhook component displayed on canvas [-]

12.2 Webhook URL generated automatically [-]

12.3 POST endpoint accepts JSON and plain-text bodies returning 202 [x]

13. Core Components — Agent

13.1 Agent component displayed on canvas with default settings [x]

13.2 Connecting an external model in Agent drops the prior model selection [x]

13.3 Flow with Agent saved and reopened — settings persist [ ]

13.4 max_iterations limits agent cycles [ ]

13.5 max_tokens limits response size [ ]

13.6 reasoning_effort field is conditional on the model [ ]

13.7 Agent Instructions (system prompt) is respected [ ]

13.8 Input via direct field vs handle (ChatInput) [ ]

13.9 Structured output schema generates valid JSON [ ]

13.10 Empty response or model refusal — no crash [ ]

13.11 Toggle add_current_date_tool works [ ]

13.12 handle_parsing_errors controls behavior on parse failure [ ]

13.13 Image via input handle is processed by Agent [ ]

14. Authentication — Login and Logout

14.1 Login with valid credentials [-]

1.1.a GET `/health_check` → status 200 `[-]`

1.1.b GET `/api/v1/version` → returns version, main_version, package `[x]`

2.1 POST `/api/v1/flows/` → creates flow, returns ID `[-]`

2.2 GET `/api/v1/flows/` → lists user flows `[-]`

2.3 GET `/api/v1/flows/{id}` → returns flow by ID `[-]`

2.4 PATCH `/api/v1/flows/{id}` → updates name/description `[-]`

2.5 DELETE `/api/v1/flows/{id}` → removes flow, returns 200 `[-]`

2.6 GET after DELETE → should return 404 `[-]`

3.1 POST `/api/v1/run/{flow_id}` with `input_value` `[-]`

3.2 POST with `tweaks` → parameters override configuration `[-]`

3.3 POST with custom `session_id` `[-]`

3.4 POST with invalid API key → returns 401/403 `[-]`

3.5 POST to non-existent flow → returns 404 `[-]`

4.1 GET `/api/v1/all` → lists available components `[-]`

4.2 GET `/api/v1/monitor/messages` → returns array `[-]`

4.3 GET with session_id filter `[-]`

5.1 Generate curl for execution `[x]`

5.2 Generate Python code for integration `[x]`

6.1 Open component advanced options `[-]`

6.2 Edit text field (input) `[-]`

6.3 Edit dropdown `[-]`

6.4 Edit toggle `[-]`

6.5 Edit float / int field `[-]`

6.6 Edit slider `[-]`

7.1 Enable Tool Mode on a component `[-]`

7.2 Group components in Tool Mode `[-]`

8.1 Outdated component notification `[-]`

8.2 Update component action `[-]`

9.1 ChatInput receives user message `[-]`

9.2 ChatOutput displays LLM response `[-]`

10.1 Prompt with variables in curly braces `[x]`

10.2 Removing variable from prompt deletes corresponding port `[x]`

11.1 Renders on canvas with URL and API Response handles `[x]`

11.2 Inspector accepts URL and HTTP method values `[x]`

11.3 Invalid URL is accepted by field but run shows error notification `[x]`

11.4–11.8 Execute GET / POST / PUT / PATCH / DELETE returns 200 `[x]`

11.9 Non-2xx HTTP response (404) propagates as status_code without crashing `[x]`

11.10 Query parameters embedded in URL are sent and echoed `[x]`

11.11 Headers table accepts key + value cell entries via inspector `[x]`

11.12 cURL tab switches mode and field accepts a cURL command `[x]`

11.13 cURL parser auto-fills URL field and executes the GET, returning 200 `[x]`

12.1 Webhook component displayed on canvas `[-]`

12.2 Webhook URL generated automatically `[-]`

12.3 POST endpoint accepts JSON and plain-text bodies returning 202 `[x]`

13.1 Agent component displayed on canvas with default settings `[x]`

13.2 Connecting an external model in Agent drops the prior model selection `[x]`

13.3 Flow with Agent saved and reopened — settings persist `[ ]`

13.4 max_iterations limits agent cycles `[ ]`

13.5 max_tokens limits response size `[ ]`

13.6 reasoning_effort field is conditional on the model `[ ]`

13.7 Agent Instructions (system prompt) is respected `[ ]`

13.8 Input via direct field vs handle (ChatInput) `[ ]`

13.9 Structured output schema generates valid JSON `[ ]`

13.10 Empty response or model refusal — no crash `[ ]`

13.11 Toggle add_current_date_tool works `[ ]`

13.12 handle_parsing_errors controls behavior on parse failure `[ ]`

13.13 Image via input handle is processed by Agent `[ ]`

14.1 Login with valid credentials `[-]`

14.2 Login with invalid credentials `[-]`

14.3 Logout redirects to login screen `[-]`

14.4 Auto-login enabled — skips login screen `[-]`

14.5 Auto-login disabled — displays login screen `[-]`

14.6 Expired session — redirects to login `[-]`

14.7 Session cleanup after logout `[-]`

15.1 Admin creates new user `[-]`

15.2 Admin deactivates user `[-]`

15.3 Admin activates inactive user `[-]`

15.4 Admin renames user `[-]`

15.5 Admin changes user password `[-]`

15.6 Isolation: user A cannot see user B's flows `[-]`

16.1 Create global variable from Settings page `[x]`

16.2 Edit existing global variable `[x]`

16.3 Delete global variable `[x]`

16.4 Create global variable of type "Generic" `[x]`

16.5 Credential variable value is hidden from the variable list `[x]`

17.1 Upload file via component `[-]`

17.2 Upload files of different types `[-]`

17.3 File size limit `[-]`

18.1 Agent responds without connected tools `[x]`

18.2 Agent displays reasoning steps and returns valid response `[x]`