feat(playground): content blocks frontend renderer for v2 workflows (#13391)

* feat: native v2 workflows endpoint with pluggable stream protocols

Rebased onto release-1.10.0. The base independently rebuilt the v2
workflows backend (RBAC, body globals, share-aware fetch); keep our
forward design and conform its auth to that work:

1. Auth: keep get_current_user_for_workflow (session-or-API-key authN
   that does not hold a DB connection during the inline run, avoiding
   the SQLite lock contention api_key_security would cause) and enforce
   the base's RBAC on top: ensure_flow_permission(EXECUTE) before run,
   (READ) before status reconstruct, with widen_for_shares fetch.
2. Port the base's request-body globals onto the v2 WorkflowRunRequest.
   The X-LANGFLOW-GLOBAL-VAR-* headers stay supported (the Responses API
   passes globals that way); body globals win on conflict. Converters
   echo the effective globals via effective_globals.
3. Public endpoint keeps the v1 build_public_tmp posture
   (access_type==PUBLIC, run-as-owner); RBAC applies to the
   authenticated endpoint only.
4. Preserve the base's post-build KB-cache invalidation in the AG-UI
   build path.

The endpoint, AG-UI bridge, pluggable stream adapters, public endpoint,
and re-attach are unchanged.

* feat(api): add output_text and session_id to v2 workflow response

The synchronous /api/v2/workflows response keyed every result under its
component id, so reading the answer meant knowing an id you can't predict.
Surface two additive fields:

- output_text: the flow's single text answer (ChatOutput/TextOutput). None
  when the flow has zero or multiple text outputs, so callers read outputs
  rather than the shortcut guessing which channel is the answer.
- session_id: echoes the resolved session so chat/memory callers can
  continue the same thread (v1 /run returned this; v2 had dropped it).

outputs is unchanged, so this is non-breaking.

* test(api/v2): cover output_text and session_id on the v2 workflow response

Pin the sync-response shortcuts on the v2 workflows endpoint:
- output_text surfaces the lone ChatOutput/TextOutput text and stays None for
  non-output message nodes, data-only flows, and multi-text flows
- session_id echoes the resolved session; the error response exposes neither
- each outputs entry exposes only {type, status, content, metadata}, with the
  component id carried by the dict key

Also drop the component_id kwarg the converter passed to ComponentOutput, which
has no such field and silently dropped it.

* feat(api/v2): structured output with resolution reason on v2 response

Replace the flat output_text shortcut with an `output` object carrying the
resolved text answer plus a `reason` that explains why it resolved that way
(single/multiple/none/non_string/failed), so a null answer is always
diagnosable instead of silently None. `reason` follows the LLM-domain
finish_reason/stop_reason convention, distinct from the lifecycle status.

Also add `display_name` to each ComponentOutput (the stable component id
stays the dict key) and a computed `has_errors` flag derived from errors.

* feat(api/v2): add request-side output selection (output_ids)

Let a sync caller name the output(s) they want via output_ids so
output.text resolves deterministically (reason=single) on multi-output
flows instead of going null. Selection is steer-only: it picks the
answer among the named outputs without filtering the outputs map.

Invalid ids are rejected with 422 before the flow runs (and before any
job row is created), so a typo costs no compute. Resolution considers
selected outputs that actually fired, so branching flows resolve to
whichever candidate ran.

* feat(api/v2): emit per-output events on the langflow stream

Give v2-workflows sync and the langflow stream protocol one parser. The
stream now emits a normalized "output" event per terminal output carrying
an OutputEvent (the ComponentOutput shape sync returns in outputs[id], plus
component_id). A shared build_component_output() backs both the sync
converter and the adapter, and the build loop ships authoritative vertex
metadata as an additive output_meta key on end_vertex (existing consumers
read build_data and ignore it).

This is access-pattern parity (one parser, same fields, same terminal set),
not byte-identical content: the stream reuses the v1 build path whose
display serialization differs from sync's run_graph output.

* feat: make content_blocks the source of truth for Message content

Migrate Message.text from a Pydantic field to a @computed_field over
content_blocks, and unify ContentBlock into the discriminated ContentType
union so a Message's payload is one uniform shape.

Schema changes:
- Add 7 new content types (Image, Audio, Video, File, Reasoning, Usage,
  Citation) with validators for media sources, non-negative tokens, and
  ordered citation indices
- Promote 'contents: list[ContentType]' to BaseContent so any node can
  nest (multimodal tool outputs, multi-step reasoning, grouped errors)
- Fold ContentBlock into BaseContent and into the ContentType union with
  tag 'group'; content_blocks is now 'list[ContentType]' everywhere
- Fix Data.__setattr__ to route through property descriptors via MRO walk

Setter / serialization:
- text setter appends a single TextContent at the end of content_blocks,
  preserving non-text blocks in chronological order (tool calls first,
  final text last)
- model_post_init preserves explicit None in data['text'] when no
  TextContent exists in content_blocks, so callers can still distinguish
  'text was never set' from 'text was set to empty string'

from_lc_message:
- Handle AIMessage tool_calls and usage_metadata regardless of whether
  content is a string or a list (tool-calling agents commonly emit
  content='' alongside tool_calls)
- Tolerate explicit source=None in multimodal image payloads

MessageResponse.from_message / MessageTable.from_message:
- Accept any of (data['text'] set, text_stream pending, content_blocks
  non-empty) as 'content present', so tool-call-only and media-only
  messages persist rather than getting rejected as missing required
  fields

Tests cover all new content types, the unified ContentType union, the
text/content_blocks contract, the setter's chronological append, and the
required-fields gate.

* feat: stable id on content blocks + plumb LangChain tool_call_id

Adds an optional 'id: str | None' field to BaseContent for stable
identity across re-emissions of the same logical block. Producers
that have a natural id (LangChain tool_call_id, external API id, a
UUID stamped before the first emission) set it; consumers use it for
dedup and cross-frame correlation. Without an id, consumers fall back
to position-derived dedup, which assumes content_blocks is append-only
within a message lifetime.

Plumbs LangChain's 'tool_call_id' through 'Message.from_lc_message'
into 'ToolContent.id'. The same logical tool call across start, args
streaming, and result lifecycle now carries the same id, so a
re-fired add_message dedups to one ToolContent instead of producing
duplicates.

Tests cover id default/round-trip/inheritance across every concrete
content type, plus tool_call_id stability across repeated conversion,
multiple tool calls each keeping their own id, and tool_calls
alongside string content.

* fix(schema): MessageResponse parses microsecond timestamps and ContentBlock partial updates preserve unset fields

Two schema regressions surfaced in QA across the content-blocks chain:

1. MessageResponse.timestamp was typed as a bare datetime, but
   Message.timestamp default is a string with microsecond precision and
   a UTC timezone label ('%Y-%m-%d %H:%M:%S.%f %Z') that Pydantic's
   default datetime parser rejects. Any freshly built Message routed
   through MessageResponse.from_message raised ValidationError. Reuse
   the shared str_to_timestamp_validator so MessageResponse accepts
   every format Message itself recognises.

2. ContentBlock.__init__ marked every field as model_fields_set, not
   just the discriminator. The override defeated exclude_unset for the
   group content type: a patch like ContentBlock(title='new') dumped
   every defaulted field and, when merged onto an existing block by
   aupdate_messages, overwrote fields the caller never touched. Mark
   only 'type' (the discriminator) so partial updates carry the variant
   tag without clobbering the rest.

Adds regression tests in test_message_content_blocks.py: from_message
round-trips Message.timestamp without crashing, and ContentBlock
exclude_unset stays narrow to the explicit fields plus the
discriminator.

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* feat(playground): content blocks frontend renderer for v2 workflows

* feat(frontend): add v2 (beta) tab to the API Access modal

Add a v1 / v2 (beta) version toggle above the language tabs. The v2 tab
emits Python, JavaScript, and cURL snippets for POST /api/v2/workflows.

The two examples are framed by outcome, not by API jargon: "Get the full
result" (the default single JSON response) and "Stream the result as it
runs", each with a one-line plain-language description. The streaming
snippets consume the default langflow protocol (switch on the event field;
handle add_message, token, and end) rather than forcing the agui protocol.
The API key is read from an env var, and a short response peek shows the
shape a caller gets back.

* feat(frontend): v2 examples read the answer from output.text

* feat(agent): interleaved text + tool_use rendering and tabbed tool-output visualizer

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* fix(api/v2): buffer parallel messages in the AG-UI translator instead of dropping them

Parallel components stream tokens for different message ids interleaved.
The translator tracked a single open message: the first foreign token
closed the open message and tombstoned its id, so every later event for
it was dropped and its remaining text never reached the client.

Tokens for a message that cannot take the wire now buffer until the open
message genuinely ends (its add_message finalizer), then flush in arrival
order; complete messages landing mid-stream buffer the same way instead
of interleaving a second START. end/error drain all buffers before the
terminal event. The wire still carries at most one open text message, so
the stream stays AG-UI-conformant.

* [autofix.ci] apply automated fixes

* fix(api/v2): gate AG-UI message finalization on non-partial state and purge removed buffers

A partial add_message re-fire (the agent path emits these at tool
start/end for a message it is still streaming) was treated as the
finalizer: it closed and tombstoned the id, so the post-tool answer
was dropped. Only a non-partial add_message finalizes now; state
defaults to complete, so payloads without properties are unchanged.

remove_message now purges a buffered message and tombstones its id,
so text the backend retracted is not flushed to the client later.

* fix(playground): scheme-guard content block URLs against unsafe schemes

Lift safeUrl into a shared chatComponents/url.ts and route the file
content block's download anchor through it, degrading unsafe-scheme
URLs to a non-clickable label. Gate the image/audio/video src through
the same helper so a data:text/html or attacker-chosen src is dropped
rather than rendered. Matches the existing citation sanitization in
SourcesStrip. Adds file-branch tests for an http URL and a
javascript: URL.

* refactor(playground): extract content-block layout + media renderers, drop favicon egress

- Extract resolveContentBlockLayout from the duplicated shape-detection
  block in bot-message and chat-message into chat-messages/utils, with a
  direct unit test (legacy group, interleaved flat, text-only, divergent
  text, edit mode).
- Split image/audio/video/file renderers out of ContentDisplay into
  MediaContentDisplay so the dispatcher stops growing per content type.
- Replace the Google favicon fetch in SourcesStrip with a local Globe
  icon so cited source domains aren't leaked to a third party.
- Trim WHAT comments to WHY, add typed test factories.

* feat: make content_blocks the source of truth for Message content

Migrate Message.text from a Pydantic field to a @computed_field over
content_blocks, and unify ContentBlock into the discriminated ContentType
union so a Message's payload is one uniform shape.

Schema changes:
- Add 7 new content types (Image, Audio, Video, File, Reasoning, Usage,
  Citation) with validators for media sources, non-negative tokens, and
  ordered citation indices
- Promote 'contents: list[ContentType]' to BaseContent so any node can
  nest (multimodal tool outputs, multi-step reasoning, grouped errors)
- Fold ContentBlock into BaseContent and into the ContentType union with
  tag 'group'; content_blocks is now 'list[ContentType]' everywhere
- Fix Data.__setattr__ to route through property descriptors via MRO walk

Setter / serialization:
- text setter appends a single TextContent at the end of content_blocks,
  preserving non-text blocks in chronological order (tool calls first,
  final text last)
- model_post_init preserves explicit None in data['text'] when no
  TextContent exists in content_blocks, so callers can still distinguish
  'text was never set' from 'text was set to empty string'

from_lc_message:
- Handle AIMessage tool_calls and usage_metadata regardless of whether
  content is a string or a list (tool-calling agents commonly emit
  content='' alongside tool_calls)
- Tolerate explicit source=None in multimodal image payloads

MessageResponse.from_message / MessageTable.from_message:
- Accept any of (data['text'] set, text_stream pending, content_blocks
  non-empty) as 'content present', so tool-call-only and media-only
  messages persist rather than getting rejected as missing required
  fields

Tests cover all new content types, the unified ContentType union, the
text/content_blocks contract, the setter's chronological append, and the
required-fields gate.

(cherry picked from commit 3b92500)

* feat: stable id on content blocks + plumb LangChain tool_call_id

Adds an optional 'id: str | None' field to BaseContent for stable
identity across re-emissions of the same logical block. Producers
that have a natural id (LangChain tool_call_id, external API id, a
UUID stamped before the first emission) set it; consumers use it for
dedup and cross-frame correlation. Without an id, consumers fall back
to position-derived dedup, which assumes content_blocks is append-only
within a message lifetime.

Plumbs LangChain's 'tool_call_id' through 'Message.from_lc_message'
into 'ToolContent.id'. The same logical tool call across start, args
streaming, and result lifecycle now carries the same id, so a
re-fired add_message dedups to one ToolContent instead of producing
duplicates.

Tests cover id default/round-trip/inheritance across every concrete
content type, plus tool_call_id stability across repeated conversion,
multiple tool calls each keeping their own id, and tool_calls
alongside string content.

(cherry picked from commit a3e8b40)

* fix(schema): MessageResponse parses microsecond timestamps and ContentBlock partial updates preserve unset fields

Two schema regressions surfaced in QA across the content-blocks chain:

1. MessageResponse.timestamp was typed as a bare datetime, but
   Message.timestamp default is a string with microsecond precision and
   a UTC timezone label ('%Y-%m-%d %H:%M:%S.%f %Z') that Pydantic's
   default datetime parser rejects. Any freshly built Message routed
   through MessageResponse.from_message raised ValidationError. Reuse
   the shared str_to_timestamp_validator so MessageResponse accepts
   every format Message itself recognises.

2. ContentBlock.__init__ marked every field as model_fields_set, not
   just the discriminator. The override defeated exclude_unset for the
   group content type: a patch like ContentBlock(title='new') dumped
   every defaulted field and, when merged onto an existing block by
   aupdate_messages, overwrote fields the caller never touched. Mark
   only 'type' (the discriminator) so partial updates carry the variant
   tag without clobbering the rest.

Adds regression tests in test_message_content_blocks.py: from_message
round-trips Message.timestamp without crashing, and ContentBlock
exclude_unset stays narrow to the explicit fields plus the
discriminator.

(cherry picked from commit 6f66393)

* fix(schema): address content_blocks review feedback

- sync langflow-base ContentBlock.__init__ with the lfx copy (model_fields_set
  parity) so exclude_unset no longer clobbers type; add cross-module regression test
- route MessageResponse content_blocks discriminator-first so stored flat blocks
  with contents=[] validate instead of raising
- move Message SecretStr coercion into model_post_init and drop the dead
  validate_text before-validator
- drop the no-op _fold_text_into_content_blocks validator
- log a shape-only debug line when from_lc_message drops an undecodable image
- type MessageResponse.content_blocks as list[ContentType] | None

(cherry picked from commit 6c7cec8)

* fix(agents): stop duplicating the final answer in content_blocks

With content_blocks as the source of truth, Message.text is a computed
field whose setter appends a trailing top-level TextContent. handle_on_chain_end
also appended the same answer into the Agent Steps group, so the final
answer rendered twice (assert 2 == 1 in test_multiple_events). The streaming
path already relies on the setter alone; make the non-streaming path match.

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* feat(schema): project new content_blocks back to the v1 wire shape

The in-memory Message and the v2 (AG-UI) path use the new content_blocks
union (groups tagged "group", every node carries id/contents, the agent
answer is a trailing top-level TextContent). The v1 API keeps emitting the
pre-1.11.0 shape via a pure legacy_render projection that runs only at the
v1 boundaries: the v1 read/response models, the memories endpoint, the v1
build SSE stream, the webhook events SSE stream, and the /run response and
stream. The build, webhook, and /run projections recurse so the Data mirror
(data.data.content_blocks) is projected alongside the top-level copy. v2
serializes the live Message and keeps the new shape.

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* [autofix.ci] apply automated fixes

* fix(api): keep simple_run_flow returning RunResponse, project v1 at the HTTP boundary

simple_run_flow is a shared helper, so wrapping its return in a JSONResponse to
apply the v1 content_blocks projection broke internal callers that call
.model_dump() on the result (the streaming run_flow_generator and
get_build_results). Return the RunResponse object from the helper and apply the
projection at the non-stream HTTP boundary in _run_flow_internal instead. The
streaming path already projects the end event via _project_run_event.

* [autofix.ci] apply automated fixes

* fix(frontend): route media content URLs through safeUrl

The media content type rendered <img src={url}> with the raw url, unlike
the sibling image/audio/video/file cases that guard via safeUrl. A
javascript:/data: scheme from untrusted tool output reached the img src.
Apply the same safeUrl guard and skip rendering when it returns null.

* fix(frontend): detect untyped legacy groups in content-block layout

resolveContentBlockLayout detected groups with type === "group", missing the
legacy / v1-projected "Agent Steps" group that is persisted without a type
field (just title + contents). Those untyped groups failed hasGroup and were
miscounted as flat non-text items, wrongly enabling ordering mode so the
duplicate top-level text rendered above the tools and the bubble body was
suppressed. Use the shared isGroupedBlock predicate, matching the rest of the
render pipeline. Also document the latent tool-less-group gap in
ContentBlockDisplay (a no-tool answer projects to a text-only group; the drop
is benign today but the gate keys off toolItems, not groupedBlocks).

* fix(frontend): render a group's displayable non-tool content

ContentBlockDisplay gated entirely on a group's tool_use leaves, so a group
whose contents had no tool_use (reasoning / citation / media, …) rendered
nothing. Collect a group's displayable non-tool leaves and render them through
the same loose renderer as top-level flat leaves, while keeping text and usage
out so the legacy v1 Input/Output scaffolding stays hidden (the bubble body
already paints the answer). Adds collectGroupLooseLeaves and unit tests.

* test(frontend): deep probes for ContentBlockDisplay group rendering

Renders the real ContentBlockDisplay (real ContentDisplay / ToolCallCard /
SourcesStrip / accordion; only ESM infra is mocked) and asserts the DOM for
each content_blocks shape: a tool-less group's citation and media now render,
an untyped legacy group renders its non-tool content, a tool-bearing group
renders both the tool and its extra leaf, the legacy Input/Output text stays
hidden, text-only and usage-only groups render nothing, and the flat shape is
unchanged. Confirmed RED on the pre-fix component (4/8 fail) and GREEN after.

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* chore: re-trigger CI after [skip ci] bake commit

* fix: regenerate component_index.json after agent.py content_blocks change

The merged index still carried AgentComponent's pre-#13390 code, so the
runtime hash allow-list (built from the index) didn't match the live
agent.py. That failed the custom-component admin-only known-template
carve-out (403 instead of 200) and drifted Update Component Index.
Rebuilt via 'make build_component_index'; sha256 now 070077b2.

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Author: 3 people

src/backend/base/langflow/api/v1/openai_responses.py

@@ -513,14 +513,22 @@ async def openai_stream_generator() -> AsyncGenerator[str, None]:
                                     break
 
                         if hasattr(component_output, "results") and component_output.results:
-                            for blocks in component_output.results.get("message", {}).content_blocks:
+                            message = component_output.results.get("message")
+                            for block in getattr(message, "content_blocks", None) or []:
+                                # The agent's flat log carries tool_use as top-level
+                                # ToolContent leaves; the legacy/grouped shape nests
+                                # them inside a group's ``contents``. Handle both.
+                                if isinstance(block, ToolContent):
+                                    leaves = [block]
+                                else:
+                                    leaves = getattr(block, "contents", None) or []
                                 tool_calls.extend(
                                     {
                                         "name": content.name,
                                         "input": content.tool_input,
                                         "output": content.output,
                                     }
-                                    for content in blocks.contents
+                                    for content in leaves
                                     if isinstance(content, ToolContent)
                                 )
                     if output_text: