Skip to content

Commit f85a32a

Browse files
feat: native v2 workflows endpoint with pluggable stream protocols (langflow-ai#13307)
* feat: native v2 workflows endpoint with pluggable stream protocols Rebased onto release-1.10.0. The base independently rebuilt the v2 workflows backend (RBAC, body globals, share-aware fetch); keep our forward design and conform its auth to that work: 1. Auth: keep get_current_user_for_workflow (session-or-API-key authN that does not hold a DB connection during the inline run, avoiding the SQLite lock contention api_key_security would cause) and enforce the base's RBAC on top: ensure_flow_permission(EXECUTE) before run, (READ) before status reconstruct, with widen_for_shares fetch. 2. Port the base's request-body globals onto the v2 WorkflowRunRequest. The X-LANGFLOW-GLOBAL-VAR-* headers stay supported (the Responses API passes globals that way); body globals win on conflict. Converters echo the effective globals via effective_globals. 3. Public endpoint keeps the v1 build_public_tmp posture (access_type==PUBLIC, run-as-owner); RBAC applies to the authenticated endpoint only. 4. Preserve the base's post-build KB-cache invalidation in the AG-UI build path. The endpoint, AG-UI bridge, pluggable stream adapters, public endpoint, and re-attach are unchanged. * feat(api): add output_text and session_id to v2 workflow response The synchronous /api/v2/workflows response keyed every result under its component id, so reading the answer meant knowing an id you can't predict. Surface two additive fields: - output_text: the flow's single text answer (ChatOutput/TextOutput). None when the flow has zero or multiple text outputs, so callers read outputs rather than the shortcut guessing which channel is the answer. - session_id: echoes the resolved session so chat/memory callers can continue the same thread (v1 /run returned this; v2 had dropped it). outputs is unchanged, so this is non-breaking. * test(api/v2): cover output_text and session_id on the v2 workflow response Pin the sync-response shortcuts on the v2 workflows endpoint: - output_text surfaces the lone ChatOutput/TextOutput text and stays None for non-output message nodes, data-only flows, and multi-text flows - session_id echoes the resolved session; the error response exposes neither - each outputs entry exposes only {type, status, content, metadata}, with the component id carried by the dict key Also drop the component_id kwarg the converter passed to ComponentOutput, which has no such field and silently dropped it. * feat(api/v2): structured output with resolution reason on v2 response Replace the flat output_text shortcut with an `output` object carrying the resolved text answer plus a `reason` that explains why it resolved that way (single/multiple/none/non_string/failed), so a null answer is always diagnosable instead of silently None. `reason` follows the LLM-domain finish_reason/stop_reason convention, distinct from the lifecycle status. Also add `display_name` to each ComponentOutput (the stable component id stays the dict key) and a computed `has_errors` flag derived from errors. * feat(api/v2): add request-side output selection (output_ids) Let a sync caller name the output(s) they want via output_ids so output.text resolves deterministically (reason=single) on multi-output flows instead of going null. Selection is steer-only: it picks the answer among the named outputs without filtering the outputs map. Invalid ids are rejected with 422 before the flow runs (and before any job row is created), so a typo costs no compute. Resolution considers selected outputs that actually fired, so branching flows resolve to whichever candidate ran. * feat(api/v2): emit per-output events on the langflow stream Give v2-workflows sync and the langflow stream protocol one parser. The stream now emits a normalized "output" event per terminal output carrying an OutputEvent (the ComponentOutput shape sync returns in outputs[id], plus component_id). A shared build_component_output() backs both the sync converter and the adapter, and the build loop ships authoritative vertex metadata as an additive output_meta key on end_vertex (existing consumers read build_data and ignore it). This is access-pattern parity (one parser, same fields, same terminal set), not byte-identical content: the stream reuses the v1 build path whose display serialization differs from sync's run_graph output. * fix(api/v2): enforce no-code-execution gate on public workflow endpoint The v2 public endpoint only ran validate_flow_for_current_settings and skipped validate_public_flow_no_code_execution, which the v1 build_public_tmp path applies. A public flow containing a Python interpreter/REPL (or the legacy Python Code Structured tool, Smart Transform lambda) was therefore an unauthenticated server-side code-execution primitive (report H1-3754930). Mirror v1: import the validator and call it right after the public-access gate. PublicFlowValidationError subclasses CustomComponentValidationError, so the existing handler already sanitizes it to a 400 'This flow cannot be executed.' without leaking the blocked component class names. Add a non-mocking test that builds a public flow with a real PythonREPLComponent and asserts the sanitized 400 (verified RED: returns 200 without the gate). LE-1389 * fix(api/v2): reconstruct background workflow status from job-keyed vertex builds A completed background job's GET status 500'd with 'No vertex builds found for job_id'. The background build path differed from the sync path twice: 1. generate_flow_events minted a fresh run_id instead of using job_id, so vertex builds were keyed by an id the status query never uses. Thread run_id through _stream_event_frames -> generate_flow_events and pass job_id from the background buffer so graph.run_id == job_id (the sync path already does graph.set_run_id(job_id)). 2. The SSE build loop (build_vertices) only persisted builds when log_builds was set and never passed job_id. Tie log_builds to job-tracked runs (run_id present) and pass job_id=graph.run_id on the persist call. Job-tracked runs also persist streaming terminal vertices so reconstruction is complete; the live build path (run_id is None) keeps its original behavior, so the v1 build path is unchanged. Test: a real background run polled to completion, then GET status asserts a reconstructed 200 (verified RED: 500 'No vertex builds found' before the fix). Covers the non-streaming flow. v1 build path unchanged (35 build tests pass); AG-UI suite 46 pass. LE-1389 * fix(api/v2): merge workflow AG-UI cancellation hardening LE-1389 * fix(api/v2): signal cross-worker workflow stops LE-1389 * fix(api/v2): report unconfirmed workflow stops LE-1389 * fix(api/v2): keep background workflows out of polling watchdog LE-1389 * fix(api/v2): buffer parallel messages in the AG-UI translator instead of dropping them Parallel components stream tokens for different message ids interleaved. The translator tracked a single open message: the first foreign token closed the open message and tombstoned its id, so every later event for it was dropped and its remaining text never reached the client. Tokens for a message that cannot take the wire now buffer until the open message genuinely ends (its add_message finalizer), then flush in arrival order; complete messages landing mid-stream buffer the same way instead of interleaving a second START. end/error drain all buffers before the terminal event. The wire still carries at most one open text message, so the stream stays AG-UI-conformant. * fix(api/v2): gate AG-UI message finalization on non-partial state and purge removed buffers A partial add_message re-fire (the agent path emits these at tool start/end for a message it is still streaming) was treated as the finalizer: it closed and tombstoned the id, so the post-tool answer was dropped. Only a non-partial add_message finalizes now; state defaults to complete, so payloads without properties are unchanged. remove_message now purges a buffered message and tombstones its id, so text the backend retracted is not flushed to the client later. * Fix AG-UI workflow lifecycle edges * [autofix.ci] apply automated fixes * fix(frontend): enable downlevelIteration for jest Set/Map iteration ts-jest compiles with target es5; without downlevelIteration, [...set] and for...of over a Set/Map emit ES5 that yields nothing. That silently broke the AG-UI bridge tests: runningNodeIds spread, markRunningNodesFailed, and restoreOriginalBuildStatuses all iterated empty. Production (Vite/SWC, modern target) was never affected; only the ts-jest harness was. Fixes the 3 failing jest tests on this branch with no other suite changes (4994/4994 pass). * fix(api/v2): surface inactivated branch vertices over AG-UI A branch component (If-Else, Conditional Router) reports its not-taken vertices in build_data.inactivated_vertices, but the AG-UI translator only emitted the branch node's own success/error status and dropped that list. The canvas seeds every planned node as pending from vertices_sorted; skipped vertices then get no build_start/end_vertex, so they stayed stuck on pending instead of rendering as inactive (the v1 build path marked them INACTIVE). The translator now appends an inactive STATE_DELTA op per inactivated vertex, and the frontend bridge maps the new inactive status to BuildStatus.INACTIVE and tears its edges down like a completed node. Fixes the If-Else regression in general-bugs-reset-flow-run.spec.ts. * fix(api/v2): dedupe repeated inactive node deltas in AG-UI stream build.py keeps reporting a conditionally-excluded vertex in inactivated_vertices on every subsequent end_vertex (the excluded set persists until the ConditionalRouter clears it), so the translator was putting the same inactive STATE_DELTA on the wire once per remaining vertex. Track emitted inactive nodes and skip re-emitting; drop a node from the set when it actually runs again (build_start/end_vertex) so a loop re-activation can still re-emit inactive later. * fix(api/v2): address review findings on the v2 workflows endpoint - recover session_id for completed background jobs from the persisted terminal message instead of always returning null, so GET status can continue the same chat/memory thread - replay a user-cancel as a CUSTOM cancel marker + RUN_FINISHED (agui) and a `cancelled` terminal (langflow) instead of RUN_ERROR, so a re-attaching client no longer reads a deliberate stop as a failure - cancel the evicted still-running buffer writer when the background-run registry is full, so it stops appending into a run no reader can find - derive per-component status from the error artifact / valid flag instead of hardcoding COMPLETED, and stop the langflow adapter dropping `valid` - throttle the unauthenticated public endpoint per IP and bound its input_value/session_id length - document the sync-only scope of request-body globals - document that live event re-attach is intentionally owner-only * test(lfx): register public_flow_rate_limit_per_minute in settings composition * refactor(v2 workflows): split workflow.py and address review blockers Splits the ~1.5k-line workflow.py into focused modules and folds in the execution-timeout and error-sanitization fixes from Cristhianzl's review of langflow-ai#13307. - B1: workflow.py now holds only the four route handlers. Validation guards move to workflow_validation, the sync/stream run loop to workflow_execution, and the durable background machinery to workflow_background (layered, acyclic). - I1: add workflow_execution_timeout (default 300) and apply a single wall-clock ceiling across sync, stream, background, and public via _stream_event_frames. A timeout becomes a sanitized terminal error and marks a background job failed. - I3: the route error handlers no longer echo raw exception text. They return a generic, code-tagged message and log the full exception server-side. - R1: remove the "commented out / future scope" comments that sat over live dataframe-extraction code in converters.py. - R4: drop the worker-routing internals from the reattach 409 message. Tests cover the timeout terminal-error path and the error-body sanitization, and the settings field-count guard is updated for the new setting. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1 parent 2b7b113 commit f85a32a

52 files changed

Lines changed: 11391 additions & 2686 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

src/backend/base/langflow/api/build.py

Lines changed: 49 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,32 @@
5555
STREAMING_ACTIVITY_REFRESH_S = 10.0
5656

5757

58+
def _output_meta_for_vertex(graph: Graph, vertex_id: str) -> dict:
59+
"""Authoritative per-output metadata for the v2 ``output`` stream event.
60+
61+
Sourced from the real graph vertex (the only place ``display_name`` /
62+
``is_output`` / declared output types are authoritative) so the streamed
63+
``OutputEvent`` matches the sync ``outputs[id]`` ``ComponentOutput``. Shipped as
64+
an additive ``output_meta`` key on ``end_vertex``; existing consumers read
65+
``build_data`` and ignore this. ``is_terminal`` mirrors the sync ``outputs`` set
66+
so the stream emits an ``output`` event for exactly the same components.
67+
"""
68+
vertex = graph.get_vertex(vertex_id)
69+
output_types = vertex.outputs[0].get("types", []) if (vertex.outputs and len(vertex.outputs) > 0) else []
70+
try:
71+
terminal_ids = set(graph.get_terminal_nodes())
72+
except AttributeError:
73+
terminal_ids = {v.id for v in graph.vertices if not graph.successor_map.get(v.id, [])}
74+
return {
75+
"component_id": vertex.id,
76+
"display_name": vertex.display_name or vertex.vertex_type,
77+
"vertex_type": vertex.vertex_type,
78+
"is_output": bool(vertex.is_output),
79+
"is_terminal": vertex_id in terminal_ids,
80+
"output_types": output_types,
81+
}
82+
83+
5884
def _log_component_input_telemetry(
5985
vertex,
6086
vertex_id: str,
@@ -337,13 +363,19 @@ async def generate_flow_events(
337363
current_user: CurrentActiveUser,
338364
flow_name: str | None = None,
339365
source_flow_id: uuid.UUID | None = None,
366+
run_id: str | None = None,
367+
track_job_status: bool = True,
340368
) -> None:
341369
"""Generate events for flow building process.
342370
343371
This function handles the core flow building logic and generates appropriate events:
344372
- Building and validating the graph
345373
- Processing vertices
346374
- Handling errors and cleanup
375+
376+
When ``run_id`` is provided the graph adopts it instead of minting a fresh
377+
one, so callers (e.g. background jobs) can later look up the run's vertex
378+
builds by that id. Defaults to a fresh uuid for the live build path.
347379
"""
348380
chat_service = get_chat_service()
349381
telemetry_service = get_telemetry_service()
@@ -354,14 +386,14 @@ async def build_graph_and_get_order() -> tuple[list[str], list[str], Graph]:
354386
start_time = time.perf_counter()
355387
components_count = 0
356388
graph = None
357-
run_id = str(uuid.uuid4())
389+
build_run_id = run_id or str(uuid.uuid4())
358390
try:
359391
flow_id_str = str(flow_id)
360392
# Create a fresh session for database operations
361393
async with session_scope() as fresh_session:
362394
graph = await create_graph(fresh_session, flow_id_str, flow_name)
363395

364-
graph.set_run_id(run_id)
396+
graph.set_run_id(build_run_id)
365397
first_layer = sort_vertices(graph)
366398

367399
for vertex_id in first_layer:
@@ -374,13 +406,13 @@ async def build_graph_and_get_order() -> tuple[list[str], list[str], Graph]:
374406
vertices_to_run = list(graph.vertices_to_run.union(get_top_level_vertices(graph, graph.vertices_to_run)))
375407

376408
await chat_service.set_cache(flow_id_str, graph)
377-
await log_telemetry(start_time, components_count, run_id=run_id, success=True)
409+
await log_telemetry(start_time, components_count, run_id=build_run_id, success=True)
378410

379411
except Exception as exc:
380412
await log_telemetry(
381413
start_time,
382414
components_count,
383-
run_id=run_id,
415+
run_id=build_run_id,
384416
success=False,
385417
error_message=str(exc),
386418
)
@@ -497,8 +529,12 @@ async def _build_vertex(vertex_id: str, graph: Graph, event_manager: EventManage
497529

498530
result_data_response.message = artifacts
499531

500-
# Log the vertex build
501-
if not vertex.will_stream and log_builds:
532+
# Log the vertex build. Job-tracked runs (background workflows pass a
533+
# ``run_id``) persist every vertex, including streaming terminal outputs,
534+
# so GET-status reconstruction by job_id is complete. The live build
535+
# path (``run_id is None``) keeps the original "skip streaming vertices"
536+
# behavior unchanged.
537+
if log_builds and (run_id is not None or not vertex.will_stream):
502538
background_tasks.add_task(
503539
log_vertex_build,
504540
flow_id=flow_id_str,
@@ -507,6 +543,9 @@ async def _build_vertex(vertex_id: str, graph: Graph, event_manager: EventManage
507543
params=params,
508544
data=result_data_response,
509545
artifacts=artifacts,
546+
# Key the persisted build by the run id so job-tracked runs can
547+
# reconstruct status by job_id.
548+
job_id=graph.run_id,
510549
)
511550
else:
512551
await chat_service.set_cache(flow_id_str, graph)
@@ -615,7 +654,9 @@ async def build_vertices(
615654
msg = f"Error serializing vertex build response: {exc}"
616655
raise ValueError(msg) from exc
617656

618-
event_manager.on_end_vertex(data={"build_data": build_data})
657+
event_manager.on_end_vertex(
658+
data={"build_data": build_data, "output_meta": _output_meta_for_vertex(graph, vertex_id)}
659+
)
619660

620661
if vertex_build_response.valid and vertex_build_response.next_vertices_ids:
621662
tasks = []
@@ -648,7 +689,7 @@ async def build_vertices(
648689
_build_run_id: uuid.UUID | None = None
649690
try:
650691
_build_run_id = uuid.UUID(graph.run_id) if graph.run_id else None
651-
if _build_run_id is not None:
692+
if track_job_status and _build_run_id is not None:
652693
_build_job_svc = get_job_service()
653694
await _build_job_svc.create_job(
654695
job_id=_build_run_id,

src/backend/base/langflow/api/router.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
from langflow.api.v2 import files_router as files_router_v2
4040
from langflow.api.v2 import mcp_router as mcp_router_v2
4141
from langflow.api.v2 import registration_router as registration_router_v2
42+
from langflow.api.v2 import workflow_public_router as workflow_public_router_v2
4243
from langflow.api.v2 import workflow_router as workflow_router_v2
4344

4445
router_v1 = APIRouter(
@@ -124,6 +125,7 @@ def _include_agentic_router():
124125
router_v2.include_router(mcp_router_v2)
125126
router_v2.include_router(registration_router_v2)
126127
router_v2.include_router(workflow_router_v2)
128+
router_v2.include_router(workflow_public_router_v2)
127129

128130
router = APIRouter(
129131
prefix="/api",

src/backend/base/langflow/api/utils/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@
4545
build_graph_from_db_no_cache,
4646
cascade_delete_flow,
4747
scope_session_to_namespace,
48+
validate_public_files,
4849
verify_public_flow_and_get_user,
4950
)
5051

@@ -90,5 +91,6 @@
9091
"remove_api_keys",
9192
"scope_session_to_namespace",
9293
"validate_is_component",
94+
"validate_public_files",
9395
"verify_public_flow_and_get_user",
9496
]

src/backend/base/langflow/api/utils/flow_utils.py

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
from __future__ import annotations
44

5+
import re
56
import uuid
67
from typing import TYPE_CHECKING
78

@@ -131,6 +132,42 @@ async def cascade_delete_flow(session: AsyncSession, flow_id: uuid.UUID) -> None
131132
raise RuntimeError(msg, e) from e
132133

133134

135+
# Public flow file paths must be ``{source_flow_id}/{safe_basename}`` — uploads
136+
# under that namespace are the only legitimate inputs for an unauthenticated
137+
# build. Anything else (absolute paths, traversal, foreign flow_ids) is a
138+
# probe at the arbitrary-file-read class of bug (GHSA-rcjh-r59h-gq37).
139+
_PUBLIC_FILE_PATH_RE = re.compile(
140+
r"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})/([^/\\]+)$"
141+
)
142+
_PUBLIC_FILE_REJECTED_SUBSTRINGS = ("\x00", "..", "\\")
143+
144+
145+
def validate_public_files(files: list[str] | None, source_flow_id: uuid.UUID) -> None:
146+
"""Reject file references that aren't ``{source_flow_id}/{basename}``.
147+
148+
Mitigates GHSA-rcjh-r59h-gq37: an unauthenticated build must not be
149+
able to address files outside its own flow's storage namespace.
150+
Called from any endpoint that accepts caller-supplied file references
151+
under a public-access boundary.
152+
"""
153+
if not files:
154+
return
155+
expected_flow_id = str(source_flow_id).lower()
156+
for entry in files:
157+
if not isinstance(entry, str) or not entry:
158+
raise HTTPException(status_code=400, detail="Invalid file entry")
159+
if any(token in entry for token in _PUBLIC_FILE_REJECTED_SUBSTRINGS):
160+
raise HTTPException(status_code=400, detail="Invalid file path")
161+
match = _PUBLIC_FILE_PATH_RE.match(entry)
162+
if not match:
163+
raise HTTPException(status_code=400, detail="Invalid file path format")
164+
flow_id_segment, basename = match.group(1), match.group(2)
165+
if flow_id_segment.lower() != expected_flow_id:
166+
raise HTTPException(status_code=400, detail="File not in this flow's namespace")
167+
if basename in (".", ".."):
168+
raise HTTPException(status_code=400, detail="Invalid filename")
169+
170+
134171
def compute_virtual_flow_id(identifier: str | uuid.UUID, flow_id: uuid.UUID) -> uuid.UUID:
135172
"""Compute a deterministic virtual flow ID for session/message isolation.
136173

src/backend/base/langflow/api/v1/chat.py

Lines changed: 5 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
from __future__ import annotations
22

33
import asyncio
4-
import re
54
import time
65
import traceback
76
import uuid
@@ -34,6 +33,7 @@
3433
get_top_level_vertices,
3534
parse_exception,
3635
scope_session_to_namespace,
36+
validate_public_files,
3737
verify_public_flow_and_get_user,
3838
)
3939
from langflow.api.v1.schemas import (
@@ -747,34 +747,9 @@ async def build_flow_and_stream(flow_id, inputs, background_tasks, current_user)
747747
)
748748

749749

750-
# Public flow file paths must be `{source_flow_id}/{safe_basename}` — uploads
751-
# under that namespace are the only legitimate inputs for an unauthenticated
752-
# build. Anything else (absolute paths, traversal, foreign flow_ids) is a
753-
# probe at the arbitrary-file-read class of bug.
754-
_PUBLIC_FILE_PATH_RE = re.compile(
755-
r"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})/([^/\\]+)$"
756-
)
757-
_PUBLIC_FILE_REJECTED_SUBSTRINGS = ("\x00", "..", "\\")
758-
759-
760-
def _validate_public_files(files: list[str] | None, source_flow_id: uuid.UUID) -> None:
761-
"""Reject file references that aren't `{source_flow_id}/{basename}`."""
762-
if not files:
763-
return
764-
expected_flow_id = str(source_flow_id).lower()
765-
for entry in files:
766-
if not isinstance(entry, str) or not entry:
767-
raise HTTPException(status_code=400, detail="Invalid file entry")
768-
if any(token in entry for token in _PUBLIC_FILE_REJECTED_SUBSTRINGS):
769-
raise HTTPException(status_code=400, detail="Invalid file path")
770-
match = _PUBLIC_FILE_PATH_RE.match(entry)
771-
if not match:
772-
raise HTTPException(status_code=400, detail="Invalid file path format")
773-
flow_id_segment, basename = match.group(1), match.group(2)
774-
if flow_id_segment.lower() != expected_flow_id:
775-
raise HTTPException(status_code=400, detail="File not in this flow's namespace")
776-
if basename in (".", ".."):
777-
raise HTTPException(status_code=400, detail="Invalid filename")
750+
# NOTE: ``validate_public_files`` (the canonical helper that mitigates
751+
# GHSA-rcjh-r59h-gq37) was moved to ``langflow.api.utils.flow_utils`` so v2's
752+
# public workflow endpoint shares the exact same gate. Keep it imported above.
778753

779754

780755
@router.post("/build_public_tmp/{flow_id}/flow")
@@ -837,7 +812,7 @@ async def build_public_tmp(
837812
# Reject caller-supplied file references that aren't scoped to this
838813
# public flow's own storage namespace. Done before any flow lookup so
839814
# malformed requests fail fast and don't touch the DB.
840-
_validate_public_files(files, flow_id)
815+
validate_public_files(files, flow_id)
841816

842817
# Verify this is a public flow and get the associated user
843818
client_id = request.cookies.get("client_id")

src/backend/base/langflow/api/v2/__init__.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,12 @@
44
from .mcp import router as mcp_router
55
from .registration import router as registration_router
66
from .workflow import router as workflow_router
7+
from .workflow_public import router as workflow_public_router
78

8-
__all__ = ["files_router", "mcp_router", "registration_router", "workflow_router"]
9+
__all__ = [
10+
"files_router",
11+
"mcp_router",
12+
"registration_router",
13+
"workflow_public_router",
14+
"workflow_router",
15+
]

0 commit comments

Comments
 (0)