Improvements to AI Form Filler, AIOrchestrator, and MessageList styling

[yuriy-fix]

Description

Various improvements to AI integration features in Flow:

• Features for making AI-edits to form fields reviewable and revertable
• API for intercepting and modifying a prompt before it's sent to the LLM
• Optional background thread execution of LLM turns to avoid UI freezing (when used without streaming)
• Built-in style variants to make MessageList look more like typical human-chat and AI-chat UIs.
• Other QoL improvements such as metadata with token usage and finish reason

Tier

Mixed. The Form Filler and the other built-in controllers are commercial (vaadin-ai-extensions-flow). The orchestrator, the request interceptor, background execution, and the message component styles are free (vaadin-ai-core-flow and the web components).

License

Vaadin Commercial License and Service Terms for vaadin-ai-extensions-flow. Apache 2.0 for the rest.

Motivation

Background

The previous release shipped the AI Form Filler (FormAIController) alongside the Grid and Chart controllers, behind the com.vaadin.experimental.aiComponents preview flag. This update also splits that module into a free and a commercial part. The rest of the scope comes from three places:

• Planned work: source tracking with a confidence level, the prompt validation hook, and the chat styling design for the message components.
• DX test findings on the Form Filler: the highlight should be automatic, revertable, and customizable, the form should show that the AI is working, and the value change listener should report one field at a time through an event type. The field marker replaces the highlight.
• Reports from the preview community: the turn holding the session lock, the empty tool parameter schema, and the generic tool error message.

Problem

• A user cannot tell which values the AI wrote, why, or how to undo them. The application had to wire up field highlighting itself, and the highlight neither explained the fill nor offered a way to revert the value.
• A reviewer cannot check a filled value without opening the source document again.
• An application has no place to check, change, or reject a prompt before the orchestrator acts on it.
• Without streaming, the turn runs on the thread that starts it and holds the session lock. Nothing reaches the browser until the turn ends, so the application looks frozen and a progress indicator set before the call appears only when the turn is over.
• Built-in tools that take no arguments declare either no parameter schema or one with no properties. Models disagree on what to send, and some vendors reject the request, which loses the turn.
• The built-in controller tools return a fixed message for anything the application throws, so the model only learns that something failed and retries the same call. A tool the application registers itself already has its message relayed.
• Building a chat from Message List, Message, and Message Input needs a lot of custom CSS.

Solution

Per-field source tracking and a revertable marker in the Form Filler, a request interceptor and opt-in background execution in the orchestrator, a parameter schema and a relayable exception for the built-in tools, and chat styling for the message components. The module also splits into a free core and a commercial extensions part.

Requirements

Form Filler

• Mark every field the AI changed with an AI field marker: a badge with a popover that explains the fill and a control that puts back the previous value. On by default, and can be turned off or localized.
  • feat!: replace field highlight with revertable AI marker flow-components#9868
  • feat: add AI field marker web-components#12351
  • docs: document the automatic AI form filler field marker docs#5873
• While a turn runs, show a working indicator on writable fields and block edits on the client, instead of setting the fields read-only on the server.
  • feat!: replace field highlight with revertable AI marker flow-components#9868
  • feat: add AI field marker web-components#12351
• Let an application supply its own content for the marker popover.
  • feat: support custom content in AI field marker popover web-components#12439
  • feat: support custom content in AI field marker popover flow-components#9922
  • docs: document custom AI field marker popover content docs#5887
• Fire the field value change listener once per changed field, passing an event type instead of a list of changes.
  • refactor!: emit form filler field value events per field flow-components#9593
  • docs: update form filler field value change listener docs docs#5732
• Opt-in source tracking: for each value it fills, the model reports the snippets it read, where each snippet is in the document, and how sure it was. Plain Java API, so an application can render a source in any UI, store it for an audit trail, and restore it after a reload.
  • feat: report per-field sources for AI form fills flow-components#9900
  • docs: document AI form fill source tracking docs#5881
• Show the reported confidence level on the field marker. Needs source tracking, and a value the model reported without a level shows no indicator.
  • feat: show confidence level on AI field marker flow-components#9901
  • feat: add confidence indicator to AI field marker web-components#12436
• Complete the Form Filler documentation.

Orchestrator

• Request interceptor that runs before the orchestrator acts on a prompt, whether it came from a connected input component or from code. It can check or rewrite the text, replace the attachments, reject the prompt with or without a message to the user, and postpone it so long work runs on the application's own threads.
  • feat: implement AI orchestrator request interceptor API flow-components#9879
• Opt-in background execution of an LLM turn, so the turn does not hold the session lock while it runs. Running on the calling thread stays the default.
  • feat: add background execution api for llm flow-components#9914
• Document the interceptor and background execution, including the threading rules.
• Response listener should provide a metadata with token usage and finish reason (as a String)

Built-in tools

• Declare at least one property in every built-in tool's parameter schema, including the tools that take no arguments.
• Add a public exception type an application can throw from a built-in controller's callback to pass its message to the model. The fixed message stays the default.

Message components

• Chat styling for Message List, Message, Message Input, and Upload File List: message width and bubble variants, a typing indicator, one-to-one and assistant message variants, and extra slots around the input.
  • Design for message-list typing indication web-components#12418

Packaging

• Split the components into a free vaadin-ai-core-flow module and a commercial vaadin-ai-extensions-flow module holding the built-in controllers, with a development-mode license check in each controller.
  • feat!: split AI components into free and pro modules flow-components#9887
  • docs: mark built-in AI controllers as commercial features docs#5879
  • feat!: follow AI module split into core and extensions #9304

Risks, limitations and breaking changes

Risks

Source tracking costs extra output tokens on every fill and holds document snippets in the session. It is off by default for that reason.

Limitations

Background execution and a postponed prompt both need automatic server push or polling. Manual push mode is not enough, since the orchestrator never calls ui.push() itself.

A source says where a snippet sits inside a document but not which document, so source tracking supports one attachment per prompt. With several, there is no way to tell which one a location points into.

Marks and field value change events land at the end of a turn, not as each field is filled.

Breaking changes

• showFieldHighlight and hideFieldHighlight are removed from FormAIController, together with the vaadin-field-highlighter-flow module dependency. The marker itself still ships from the @vaadin/field-highlighter npm package. Fields are no longer set read-only on the server during a fill turn.
• addFieldValueChangedListener is replaced by addFieldValueChangeListener, which fires once per changed field and takes an event instead of a list of changes.
• Changes to the DOM of Message List, Message, Message Input, and Upload File List may break custom styling.
• The vaadin-ai-components-flow artifact is replaced by vaadin-ai-core-flow and vaadin-ai-extensions-flow.
• The built-in Grid, Chart, and Form controllers now need a commercial subscription, checked in development mode.
• The components stay behind the preview flag, so more breaking changes can land in any version.

Out of scope

• Source tracking for the Grid and Chart controllers.
• Showing the source location inside the original document.
• Wiring the typing indicator into the orchestrator. Only the component styling is in scope.
• Low-level vendor-specific API on LLMProvider, including direct access to the vendor's own client.
• More than one controller per orchestrator.
• A shorter path from an UploadHandler to an orchestrator attachment. Raised in the DX tests, tracked as a separate backlog item.

Materials

• DX test report on FormAIController.
• Preview feedback thread on the AI Grid: https://vaadin.com/forum/t/ai-grid-preview-feedback/179716
• Current documentation: https://vaadin.com/docs/latest/flow/ai-support

Metrics

No response

Pre-implementation checklist

• Estimated (estimate entered into Estimate custom field)
• Product Manager sign-off
• Engineering Manager sign-off

Pre-release checklist

• Documented (link to documentation provided in sub-issue or comment)
• UX/DX tests conducted and blockers addressed
• Approved for release by Product Manager

Security review

The interceptor is where an application can check or mask prompt content before it reaches the LLM. The application decides which files reach the model. A tracked source is the model's own account of what it read in them, not a checked quote.