feat: collaborative editing - #13399
Draft
HzaRashid wants to merge 26 commits into
Draft
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
✅ Migration Validation Passed All migrations follow the Expand-Contract pattern correctly. |
HzaRashid
added
ignore-for-release
DO NOT MERGE
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
Introduce CollaborationEventService with a WAL SQLite mailbox keyed by flow_id for cross-worker fanout. Register the service in deps/schema, split event types into schemas.py, and add unit tests for publish/poll, TTL, caps, and worker isolation. Fix service factory import inference for lfx flow_operations.
- Add `WS /api/v1/flows/{flow_id}/collab` and keep `api/v1` limited to routers
by moving collaboration orchestration into `api/utils/collab/`
- Implement `FlowCollaborationConnection` for session start, operation submit,
access revalidation, accepted/rejected responses, and local peer broadcast
- Add `CollaborationManager` for per-worker room membership, presence dedupe,
accepted-operation fanout, and backplane event polling with shutdown cleanup
- Introduce typed collaboration WebSocket/backplane schemas and explicit
backplane event parsing instead of `TypeAdapter`
- Apply flow operations atomically with revision checks, write authorization,
invalid persisted `flow.data` rejection, and filesystem snapshot/restore on
failed persistence
- Split client request validation from persisted graph validation via
`FlowOperationValidationError` and `FlowDataValidationError`
- Replace flow-operation `TypeAdapter` parsing with explicit operation-type
dispatch in `lfx.services.flow_operations`
- Register collaboration event service usage, wire router exports, and add
unit tests for websocket behavior, manager fanout/deduping, and flow-operation
validation paths
- Add flow operation and collaboration WebSocket message TypeScript types aligned with the backend collab protocol - Add buildFlowCollaborationWebSocketUrl and useFlowCollaboration for session.start, revision tracking, operation submit/ack, remote broadcast handling, presence updates, and reload callbacks on stale/gap/close - Add unit tests for the hook and collab WebSocket URL helper
Introduce an opt-in beta collaboration mode on the flow canvas that emits local graph operation batches over the collab WebSocket, applies remote forward_ops without full-flow autosave, and shows active collaborators. Add operation diff/adapter helpers, collaboration-aware save flushing, and coverage for drag, delete, paste/import, undo/redo, metadata, and reload boundaries, including clearing stale undo history after remote ops.
Replace snapshot undo/redo with local forward/inverse operation batches when the collaboration beta toggle is enabled, so undo submits normal collaboration operations instead of restoring whole-canvas snapshots. Gate collaboration socket, history, and operation emission behind the feature switch, and fix starter-template and drag/delete flows that were leaving the canvas empty or without undo history.
Avoid deep-cloning the full graph when a drag begins in collaboration mode. Capture just the dragged or selected node payloads needed for move undo history, which reduces drag-start work on larger flows.
Replace whole-roster WebSocket presence updates with snapshot plus incremental join/left events, and add ephemeral selection state with bootstrap snapshots and selection.updated handling. Rename the worker backplane roster event to presence.roster and simplify connection-side membership checks.
…ions and selections in a shared langflow_collaboration.sqlite database instead of merging periodic presence.roster backplane events. Session start, heartbeat, and cleanup update connection rows; snapshots and cross-worker fanout use incremental presence.joined, presence.left, and selection.updated events.
Reuse the collaboration poll loop to purge expired SQLite connections every 30s and broadcast presence.snapshot per active flow, so clients recover from missed presence.left events. Batch list_users across flow IDs, key connections by connection_id with non-null selected_at, and fold selection into presence.snapshot on the wire and frontend.
Add collaborative selection UI and keep each user's selection stable while the graph changes under collaboration beta. Frontend - presence and overlays - Add remote selection overlays (nodes, edges, participant bumps) on the canvas - Add per-node selection chrome and collaboration flow toolbar - Show collaborator avatars and connection status in CollaborationPresence - Split selection API context so PageComponent does not re-render on every presence update Frontend - selection sync - Add local selection store for immediate overlay feedback before the socket is ready - Publish selection via shared publish/resend helpers (dedupe, reconnect resend) - Clear local selection when collaboration beta is disabled or the flow changes - Sync selection on pane/node/edge clicks and React Flow selection changes Frontend - remote operations - Strip selected, measured, and dragging from operation diffs and update payloads - Preserve local node.selected when applying remote node updates - Restore edge selection after cleanEdges() rebuilds edges Frontend - edge selection geometry - Recompute edge marker positions when connected nodes move (flowNodes subscription) Backend - Assert joining a flow does not clear an existing collaborator's selection in presence snapshot Tests - Add unit tests for selection publish/sync, overlays, operation adapter, and edge transforms
…e fanout Replace per-connection presence refresh with a manager-owned heartbeat loop, and tighten collaboration connection/backplane invariants around UUIDs, worker identity, and explicit connection limits. Backend - heartbeat protocol - Add heartbeat.ping / heartbeat.pong websocket message schemas - Start a manager-owned heartbeat loop alongside the existing backplane poll loop - Stagger pings across timing-wheel buckets to avoid burst load - Track pong deadlines per connection and disconnect expired sockets - Refresh SQLite presence TTL on valid pongs - Remove per-connection _presence_heartbeat() task from FlowCollaborationConnection Backend - manager and connection lifecycle - Introduce FlowRooms with connection_id as the source of truth - Use UUID connection ids internally; only serialize at wire/storage boundaries - Enforce collaboration_max_connections at register() time, not via silent truncation - Close over-cap websocket joins with WS_1013_TRY_AGAIN_LATER - Add _registered_connection_id backing field plus connection_id property that fails loudly before session.start - Rename background task helpers to start/stop_collaboration_background_tasks Backend - backplane simplification - Add worker_id to operation.accepted backplane payloads - Ignore same-worker operation.accepted events instead of using a fanned revision cache - Remove origin_connection_id and the local (flow_id, revision) dedupe cache - Keep connection_id only for local websocket exclusion, not backplane metadata Backend - collaboration event service - Accept UUID connection ids in service APIs - Look up flow_id from connection_id inside update/remove paths - Add remove_connections() batch cleanup for expired heartbeat disconnects - Wire presence TTL from settings via CollaborationEventServiceFactory - Add CollaborationPresenceChangeEnvelope for routed presence changes Settings - Add collaboration heartbeat interval/stagger/timeout settings - Add collaboration connection TTL, max connections, and presence snapshot interval - Validate scheduler shape (stagger < interval, bucket count >= 2) Frontend - Reply to heartbeat.ping with heartbeat.pong without changing collaboration state - Add corresponding frontend protocol types and hook test coverage Tests - Add heartbeat scheduling, pong validation, limit enforcement, and settings tests - Update manager/service tests for worker_id backplane filtering and UUID connection ids - Fix websocket test helpers to share a TestClient portal for peer fanout
…dates - Replace monolithic `full_node` updates with granular `set_field`, `delete_field`, and `overwrite_node` operations to minimize conflicts in real-time collaborative editing. - Introduce `NodeFieldPath` tuple schema to accurately target nested JSON properties in node payloads. - Remove broad schema validation and deny-lists in the backend `apply.py` engine, replacing it with strict structural protocol checks and explicit operation handler routing. - Update frontend `buildGraphDiffOperations` and component hooks (e.g., `NoteNode`, `GenericNode`, `modelInputComponent`) to emit granular field patches instead of full node replacements. - Implement path-based operation coalescing and history management in `use-flow-collaboration-editing` to properly batch rapid field-level changes. - Add `flow-operation-path.ts` utilities for safe, literal path traversal and mutation in the frontend. - Add comprehensive test coverage for backend operation validation, rejecting ambiguous updates (e.g., mixing overwrite and field patches on the same node).
Convert the collaboration backplane and presence store to async APIs backed by aiosqlite so WebSocket handlers and background poll loops no longer block the event loop on SQLite I/O. Update call sites and unit tests, and add collaboration lifecycle logging for connection and presence flows.
…ping Use UUID for collaboration event ids in-process, with string conversion only at the SQLite boundary. Log backplane rebroadcasts with event_id, and replace ambiguous None states with LocalEvent and NoPendingPongDeadline sentinels for local broadcasts and heartbeat deadlines.
…ayloads Drop the server-derived actor_delegate field and FlowOperationActorDelegate enum from accepted/broadcast/backplane schemas since nothing consumed it.
… updates Drop the overwrite_node node-update op from the collaboration protocol and emit set_field/delete_field patches everywhere instead. Tighten undo/redo coalescing and remote invalidation to path-level touch detection so unrelated fields on the same node no longer clear local history.
…d ops Copy-on-write node payloads during apply so base flow data is not fully deep-copied, and reject updates/deletes targeting nodes added earlier in the same batch. Sanitize API keys in broadcast forward_ops and add tests for the apply engine and collaboration redaction paths.
Replace broad refresh/rebuild node diffs with explicit field-level update_nodes operations so concurrent local edits are preserved. Reject whole-node update paths on the backend and add regression tests for refresh, rebuild, and outputs merge behavior.
Push undo history when local operations are emitted instead of waiting for server acceptance, preserve per-user stacks across remote edits, and compute redo from the pre-undo graph so field-level changes restore the last visible value. Also allow undo/redo shortcuts from node controls and prevent model input focus/auto-select from blocking or undoing language model changes.
github-actions
Bot
added
enhancement
github-actions
Bot
added
enhancement
Contributor
|
Build successful! ✅ |
github-actions
Bot
added
enhancement
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.