Add text selection system with frame-level cell composition - #980
Conversation
Introduce a bidirectional bridge between the application and the renderer for terminal text selection. The renderer now composites each frame into a cell grid (FrameCell[][]) with per-cell selectability and semantic flow metadata, and can apply a selection highlight before serialization. New public API: - getFrameController(stdout) — access the frame controller for a render instance - FrameController.setSelection(sel) — schedule a repaint with the given selection highlighted - FrameController.getFrame() / subscribe() — read or observe the latest composited frame New <Text> props: - selectable (default true) — whether the text participates in selection - selectionFlow — group texts into a logical reading flow - selectionBreakAfter / selectionJoiner — control how line breaks are reconstructed during copy Internal changes: - Output.get() returns cells and boundaries alongside the string - renderNodeToOutput threads flow IDs and uses wrapTextWithMetadata for semantic line-break tracking - wrapTextWithMetadata wraps text and returns boundary/selectability metadata per row
Resolve the 35 lint errors blocking CI, all behavior-preserving: - Replace `null` types and runtime literals with `undefined` to match the codebase convention (@typescript-eslint/no-restricted-types). - Use `Array<T>` / `readonly T[]` per @typescript-eslint/array-type. - Use Unicode escapes with uppercase hex for the selection background sequence (unicorn/escape-case, unicorn/no-hex-escape). - Rename SELECTION_BG → selectionBackground (naming-convention). - Replace Array#reduce with an explicit loop (unicorn/no-array-reduce). - Move the public frameController field before the private fields (member-ordering) and drop an unnecessary type assertion. Generated with AI Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Add test/selection.tsx exercising the public selection API:
- getFrameController lifecycle (undefined for unknown stdout, present
after render)
- getFrame exposes composited cells with correct dimensions/values
- default vs selectable={false} cell flags
- setSelection highlights the selected region, clearing removes it, and
identical selections are deduplicated (no extra repaint)
- subscribe receives published frames
- selectionFlow groups flowIds; distinct nodes get distinct flowIds
- selectionBreakAfter="hard" records a hard boundary
Generated with AI
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
Pushed two follow-ups to get this review-ready:
@sindresorhus @vadimdemedes whenever you have time to take a look, thank you! |
|
I think the problem is worth solving, but I would not merge this PR as-is. The raw frame/controller bridge makes sense for apps that own alternate-screen input and need to implement selection themselves. Qwen Code is a good example. In normal terminal mode, native terminal selection is still simpler, so this should be scoped to the app-owned viewport use case. The main issues are:
I would split this into two PRs. First, add a small opt-in read-only frame controller for the visible alternate-screen viewport. Keep publishing internal, define the coordinate semantics, and expose only the data a consumer needs. Then add flows, boundaries, and semantic So: yes to the capability, but no to this PR in its current shape. I would simplify and narrow the first step before adding more selection semantics. |
|
Closing in favor of #984 — the first part of the split suggested in review (small opt-in read-only frame controller for the alternate-screen viewport). This branch stays around as the basis for the follow-up PR covering flows, boundaries, and semantic |
Problem
Background: why full-screen TUIs need virtual scrolling
Long-running interactive applications — coding assistants, chat clients, log viewers — accumulate content that far exceeds the terminal height. A single Qwen Code session can produce thousands of lines of conversation, tool output, and code blocks. Rendering all of this into the terminal's scrollback buffer creates two problems:
Virtual scrolling solves both: the application enters alternate-screen mode (
?1049h) to get a fixed-size viewport, then only renders the visible slice of content. As the user scrolls, the viewport window moves over the full dataset and the render output is recycled — much like a virtualized list in a GUI framework. The terminal sees a constant-height frame, eliminating flicker and enabling precise layout.The selection problem
Alternate-screen mode and virtual scrolling together break text selection, which is a basic user expectation:
The scrollback buffer is empty. The alternate screen (
?1049h) is a separate, fixed-size buffer with no scrollback history. Terminal emulators' native selection (click-drag, double-click word select, etc.) operates on the scrollback buffer. In alternate-screen mode there is nothing to select from — the content exists only as the current frame, which the terminal treats as ephemeral (by design — think vim, htop, less).Virtual scrolling means the full content is never on screen. Even if a terminal emulator supports selection within the alternate screen, it can only select what's currently rendered in the viewport. Content that has been scrolled off is not in any terminal buffer — it exists only in the application's data model. Selecting across a range that spans multiple viewport-fuls of content is impossible through terminal-native mechanisms.
Ink's render pipeline is opaque to the application. Ink composites the yoga layout tree into a string of ANSI sequences and writes it to stdout. The application has no structured access to what's actually on screen — which characters are at which cell coordinates, which are selectable, how lines wrap and flow. Without this information, the application cannot implement its own selection that maps mouse coordinates to logical text content.
The result: users of full-screen Ink TUIs cannot select and copy text — code blocks, error messages, conversation content. For a coding assistant where copying generated code is a core workflow, this is a critical usability gap.
Solution
This PR adds a frame-level cell composition system that gives applications structured, per-cell access to the rendered frame, enabling mouse-driven text selection within Ink's rendering pipeline.
The key insight: Ink already walks the yoga tree and writes styled characters into an
Outputgrid during rendering. This PR extends that grid with per-cell metadata (selectability, reading-flow grouping, line-break semantics) and exposes it through aFrameControllerbridge:The application handles mouse events and selection logic; Ink provides the composited frame data and applies the selection highlight during serialization. Selection is a pure render-time concern — it never mutates committed frame state.
Public API
getFrameController(stdout): FrameController | undefinedAccess the frame controller for a render instance (via the existing
instancesmap).FrameControllergetFrame(): ReadonlyFrame | nullgetSelection(): ScreenSelection | nullsetSelection(sel | null): voidsubscribe(listener): () => voidFrameCellEach cell in the composited grid carries:
type,value,fullWidth,styles— existing styled character dataselectable: boolean— whether this cell participates in selectionflowId: number | null— logical reading flow groupingNew
<Text>propsselectablebooleantrueselectionFlowstringselectionBreakAfter'soft' | 'hard'selectionJoinerstring''Internal changes
Output.get(selection?)— returns{output, height, cells, boundaries}. Cells carry selectable/flowId metadata. When a selection is provided, selected cells get a highlight background applied before serialization.renderNodeToOutput— threadsflowIds/nextFlowIdmaps through the tree. UseswrapTextWithMetadatainstead of plainwrapTextto track soft/hard line boundaries and per-row selectability.wrapTextWithMetadata— wraps text and returns{text, boundaries: TextBoundary[], selectableRows: boolean[]}for semantic copy reconstruction.renderer— accepts an optionalselectionparameter, creates flow tracking state, returnscells/boundariesalongside the string output.Inkclass — creates aFrameControllerin the constructor, publishes frames after each render, reads selection state for highlight.Testing
All 86 existing component tests pass. The feature is additive — no existing behavior changes when selection is not used.
This feature has been running in production in Qwen Code's TUI (via a patch on ink 7.0.3) for several months, supporting text selection in a full-screen alternate-screen TUI with virtual scrolling.