Skip to content

Epic #303: replace WebSocket/STOMP push with Server-Sent Events (client) - #316

Merged
johardi merged 14 commits into
masterfrom
epic/303-sse
Jul 27, 2026
Merged

Epic #303: replace WebSocket/STOMP push with Server-Sent Events (client)#316
johardi merged 14 commits into
masterfrom
epic/303-sse

Conversation

@johardi

@johardi johardi commented Jul 27, 2026

Copy link
Copy Markdown
Member

Closes #303 together with the companion service PRs. Thirteen atomic commits, landed via #309#314.

What this changes for the user: edits feel instant, for everyone.

Metric (same harness, same workloads) Before After
Acting user's edit → own UI updates 9,547 ms median 567 ms median
Another viewer sees the edit 9,545 ms median 307 ms median
Project-open event payload 20 KB, grows with project age forever ≤1.5 KB, constant
Project-open ready time 264 ms 269 ms

The old transport's root defect: pushed event windows carried hardcoded tags, so the client's stale-window guard dropped every push after a session's first — all delivery quietly fell back to a 10-second poll, and every project open re-downloaded the entire event archive.

This PR: gap detection with catch-up on the client (#297), open-at-head anchoring (#301), the refresh-prompt dedup fix (#300), the token-log scrub (#295), the measurement harness itself, and the centerpiece — StompJs and the polling timer replaced by a browser-native EventSource with ticket auth, browser-and-zombie-proof reconnect (liveness checks + head-carrying heartbeats), and exactly-once recovery via Last-Event-ID replay (#306).

Verification: 85/85 Playwright tests against the full epic stack, including two-viewer propagation of successive changes, offline→reconnect exactly-once, the 250-change bulk refresh prompt, and stream auth negatives.

Deployment coupling (why the e2e check is red here): the polling safety net is gone, so this client requires the SSE-era gateway/event-history/backend images. CI pins published images; once the companion epic PRs merge and release, one pin-bump commit in playwright/docker-compose.yml turns this green. Merge order: services first, this last.

Companion PRs: protegeproject/webprotege-gwt-api-gateway#50, protegeproject/webprotege-event-history-service#35, protegeproject/webprotege-backend-service#191, protegeproject/webprotege-ipc#34.

johardi added 13 commits July 24, 2026 15:15
…ject-open cost

Measures three user-perceived metrics against the running stack: how long the
acting user waits for their own edit to appear, how long a second viewer waits
for the same change, and what project open costs including the full-history
poll request. Runs via npm run test:perf with a dedicated config (one worker,
no retries) so timings are not contaminated by parallelism or hidden re-runs.

Baseline on current images: acting-user median 9.5s, propagation median 9.5s
(websocket frames arrive but are discarded due to hardcoded event tags; the
10-second poll delivers), and every project open re-downloads the entire event
history. Part of #303.
…h path

A 403 from the gateway wrote the caller's bearer token into the gwt-ui-server
log at INFO level. Anyone with log access could replay a live credential. The
denied log line keeps the user, action, and response headers, which are enough
to diagnose permission problems.

Part of #295 (#303).
The refresh prompt deduplicates signals with a handled-events set, but the
event compared only the project id, so every signal for a project after the
first looked already-handled and the prompt never fired again in a session.
Equality now includes the event id: the same signal arriving over both
delivery paths still collapses to one prompt, while each later bulk change
prompts again.

The test is JUnit 5 because this module's JUnit 4 tests do not run: the
jupiter engine is on the classpath without a vintage engine, so surefire
silently skips all 181 JUnit 4 test classes here — worth its own follow-up.

Part of #300 (#303).
…etch

The client only checked that an incoming event window was newer than its
bookmark, never that it continued from it. A window that jumped ahead was
applied anyway and the events in between were silently lost, leaving the
view stale with no signal. Now a window must start exactly at the bookmark
to be applied. A gapped or partially overlapping window is discarded and
everything from the bookmark forward is fetched through the existing pull
path, which supersedes the discarded window, so nothing is missed and
nothing is applied twice.

A single-flight guard stops a burst of gapped windows from stacking up
fetches: windows arriving mid-fetch only raise a high-water mark, and one
follow-up fetch covers the remainder if the first fetch fell short. A
failed fetch clears the guard so the next poll or push retries, and a
fetch that makes no progress does not immediately refetch, which avoids a
tight loop when the archive itself is missing the events.

With today's hardcoded push tags this check stays dormant; it becomes
load-bearing once pushes carry real sequence numbers (#296) and the
polling safety net is removed (#306).

Fixes #297 (part of #303).
…history

The events query can now carry a latestOnly flag: instead of returning
archived events it answers with an empty window positioned at the
project's current head, which is what a client needs to start receiving
live changes without downloading the past. The flag rides both hops —
Jackson for the JSON-RPC leg and the GWT-RPC custom field serializer for
the browser leg, where a missed serializer update would silently drop it.

Old servers ignore the flag and old clients never send it, so mixed
versions stay safe.

Part of #301 (part of #303).
The event bookmark started at zero on every project open, so the first
poll asked for every event ever archived for the project — an unbounded
download that grew with project age, re-dispatched years-old deltas over
freshly loaded state, and was also dumped wholesale into the server log.
The manager now starts unanchored: opening a project first requests the
current head position and only then begins normal polling, so the event
channel carries changes from the open onward and nothing else. Windows
arriving before the anchor resolves are dropped — the state queries that
run after the anchor already reflect them.

The anchor is requested before the portlet state queries are batched, so
state is always loaded at or after the anchor position and an edit
landing in between cannot vanish into the gap between history and live
delivery. A failed anchor request is retried by the next poll tick
rather than wedging the channel.

Anchoring bypasses the dispatch path deliberately: routing the empty
anchor response through it would never advance the bookmark, and against
an old server that ignores the flag the direct assignment discards the
returned history instead of replaying it.

Part of #301 (part of #303).
…tSource stream

The browser received project events two ways at once: a StompJs
websocket subscription and a ten-second polling timer backing it up.
Both are replaced by the browser's own EventSource pointed at the
gateway's SSE endpoint. The event-dispatch machinery survives the swap
intact — the stale-window guard, gap detection with catch-up, the
open-at-head anchor, and change-request awaiting all moved unchanged
into a dispatcher class that no longer owns a timer, and the catch-up
fetch is now the client's only pull path.

The stream authenticates with a short-lived project ticket minted from
a fresh bearer token, since EventSource cannot send headers. Reconnect
handling splits by failure kind: while the browser retries on its own
it resends the last event id itself and nothing needs doing; after a
fatal close, such as an expired ticket, the client mints a new ticket
and opens a fresh stream carrying the last seen id as a query
parameter, because a new EventSource never sends the header. Missed
windows in either case are recovered by gap detection. Frames arrive as
a named project-events event, so the listener binds to that name rather
than onmessage, which named events never fire.

Live updates for every viewer now ride the push channel alone — this
must not reach a deployed image before the gateway's SSE endpoint and
ticket issuance are live, since the polling safety net is gone.

Fixes #306 (part of #303).
Nothing references the CDN-loaded stomp client anymore now that project
events arrive over the browser's native EventSource.

Part of #306 (part of #303).
Every step of the open sequence — the token fetch, the ticket mint, the
connect — could fail with nothing but a log line, leaving the viewer
permanently without live updates. Most plainly: going offline makes the
browser abandon the EventSource for good, and the reopen attempted at
that moment also fails because the machine is still offline. The
end-to-end reconnect drill caught exactly that — the viewer never came
back.

A watchdog now guards every open attempt: if no stream is live ten
seconds after an attempt started, the whole chain runs again, and keeps
running until the stream opens or the project view is disposed. The
browser's own transient retries stay untouched — the watchdog only takes
over when they never land.

Part of #306 (part of #303).
With project events arriving over a permanently open EventSource, the
page never reaches the network-idle state again by definition, so every
wait for it times out. The waits become domcontentloaded; the assertions
that follow each one already poll for the UI state they need.

Part of #306 (part of #303).
A network drop can leave the EventSource half-open: the browser still
reports it connected, no error ever fires, and no frame will ever arrive
again — the reconnect drill reproduced it with a two-second offline
window. The browser cannot see the server's comment keep-alives either,
because comment frames are never delivered to page JavaScript.

The server now heartbeats with a real named event the client can
observe, and the client tracks stream activity: a stream that produces
neither a frame nor a heartbeat for two and a half heartbeat intervals
is torn down and reopened with the last seen id, so the missed window
comes back through the reconnect replay. Healthy streams just keep
re-arming the check.

Part of #306 (part of #303).
…quiet projects

A brief network loss can make the browser drop a delivered frame while
keeping the connection alive and error-free — the event is simply gone.
Gap detection only notices when a later window arrives, so on a quiet
project the loss could go undetected indefinitely.

Server heartbeats now carry the project's current head sequence, and the
client compares it with the dispatcher's bookmark on every heartbeat: a
head ahead of the bookmark means events were lost, and the existing
catch-up pull fetches the difference. Detection is bounded by the
heartbeat interval instead of by when the next edit happens to occur.

Part of #306 (part of #303).
The zombie-recovery path legitimately takes up to about a minute, past
the suite's default per-test limit.

Part of #306 (part of #303).
@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

📋 Playwright report: https://protegeproject.github.io/webprotege-gwt-ui/e2e-report/316/

Outcome of the latest run: success. Re-running this workflow overwrites the same URL.

The client no longer has a polling fallback, so the suite needs the
gateway that serves the SSE stream and tickets, the event-history
service that assigns per-project sequence numbers, and the backend with
the durable outbox. All three are now released: gateway 3.0.3,
event-history 2.0.1, backend 5.0.17.

Part of #303.
@johardi
johardi marked this pull request as ready for review July 27, 2026 22:35
@johardi
johardi merged commit 0aaf619 into master Jul 27, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Epic (alternative to #294): replace WebSocket push with Server-Sent Events (SSE)

1 participant