feat: Emulator streaming#3211
Conversation
|
here is the broker that goes into the pcsx2 container if you want to take a look at the api inside https://github.com/LoneAngelFayt/pcsx2-romm-integration |
|
I've finished setting up the integration for dolphin as well, I'm going to split it out into a different PR for each integration There is thankfully minimal changes but I turn some hardcoding into variables Once you get done looking through this I'll adapt the others and put them in as I have them completed |
9c8da66 to
6a0e5ff
Compare
|
@gantoine I changed my mind, and just added all the emulators I've gotten working along with their respective changes. I recruited Claude to help make my PR description a little better as well. Should outline everything I've touched and how it all works. |
b44b7dc to
6818507
Compare
Add /api/streaming endpoints for session claim/release, ROM launch, save states, volume control, and save-and-exit against per-platform emulator broker containers. Sessions are stored in Redis with atomic SET NX claims, owner-bound, and admin force-releasable. Config comes from config.yml (streaming.enabled + containers list) with a STREAMING_SAVE_TIMEOUT env override for slow-saving brokers. Includes 19 endpoint tests covering auth scopes, claim races, ownership, broker failure paths, and force-release.
Add a Pinia streaming store (config fetch, session lifecycle, broker controls), a v1 player view at rom/:rom/stream with save state and volume controls, and a cast button on PlayBtn for platforms with a configured streaming container. The route registers a v2 named view via v2For so the v2 fallback screen shows until a v2 player lands.
6818507 to
d94c5b6
Compare
Move the streaming player UI from v1 to v2 per the v1-frozen rule. The shared streaming store and backend stay canonical; only the frontend surface moves. - New v2 player view (Player/Stream.vue) porting the launch/playing/ error state machine, session claim, iframe UI auto-hide, volume, save/load state, fullscreen, save-and-exit and stop/release. - Fold streaming into the single Play action via useGameActions: routes to /rom/:id/stream when a container is configured, else EJS, else Ruffle. Streaming wins over in-browser emulation. - Hydrate streaming config in AppLayout (v2 users never mount v1 Main.vue, where v1 put the fetchConfig call). - Make the rom/:rom/stream route v2-only (v1 default falls back to Home). - Add stream-* i18n keys to all 18 locales. - Remove the v1 Stream player, PlayBtn cast button and Main.vue fetchConfig wiring. Generated-By: PostHog Code Task-Id: e091168f-69fd-4d4a-9f12-d473dd3f17c9
Address review findings on the streaming framework. Backend (backend/endpoints/streaming.py): - Session keys now carry a TTL (6h) so an abandoned session (broker dead / backend crashed) eventually frees the container instead of wedging it until an admin force-releases; control calls refresh the TTL so active sessions never expire. - _container_for_platform validates each entry has a scheme-bearing host, skipping malformed ones (mirroring /config) so claim/control return a clean 404 instead of a 500 KeyError on container["host"]. - _parse_host_url / _derive_broker_host reject schemeless hosts that would produce the broken "//None:8000/..." broker URL and colliding session keys; _broker_url raises a clear 502 when no usable host. - save-and-exit with wait=false drains the session (short-lived key) instead of deleting it outright, so a concurrent new claim can't /launch on top of a not-yet-dead emulator and lose the in-flight save. - New tests cover schemeless/missing-host skips, the claim TTL, and the wait=false drain window. Frontend: - stores/streaming.ts: releaseSession/saveAndExit drop the local session record only after the backend confirms, so a failed release no longer leaves the user wedged behind their own still-held session. - v2/views/Player/Stream.vue: disambiguate the 404 from claimSession (ROM not found vs container not configured) so the message matches the real failure; new stream-error-rom-not-found i18n key in all 18 locales. Generated-By: PostHog Code Task-Id: e091168f-69fd-4d4a-9f12-d473dd3f17c9
Replace English placeholder values for the stream-* keys in play.json with translations for all 16 non-English locales. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move raw HTTP calls out of the streaming store into a dedicated services/api/streaming.ts module, matching the services/api/* convention. Thin wrappers with no store state (setVolume, setMute, saveState, loadState) are dropped from the store and called directly via streamingApi in Stream.vue; the store keeps only state-managing actions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the repeated urllib request construction (URL, secret, payload, headers, urlopen, parsing) shared by the six broker functions into _broker_request and its best-effort _broker_request_safe variant. The best-effort wrappers collapse to a few lines each and the nosec B310 annotation now lives in one place. Slim the tests with a _streaming() context manager plus _auth/_claim_ok helpers, removing the repeated cm.get_config patch boilerplate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-platform slot capabilities were duplicated across the frontend store's hardcoded platformCapabilities() map and the backend's pydantic Field bounds, independent of the broker's own ceiling. Consolidate the RomM-side knowledge into one _PLATFORM_CAPABILITIES table in the streaming endpoint: - /config now ships each container's capabilities, and the frontend selector reads them instead of keeping its own copy. - The SaveState/LoadState request bounds derive from the table, and the routes validate the exact per-platform ceiling before calling the broker (clean 422 instead of a broker 502). The broker remains the ultimate enforcer; RomM no longer keeps two independent hardcoded copies of its slot semantics. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@LoneAngelFayt alright i'm happy with the state of this PR, want to give it a spin and see if everything still works? |
|
Gladly! |
|
@gantoine looks like its all working. One thing to note though. Since the v2 migration, it looks like controller inputs are being snagged by the UI as well as the stream. I fix this in a commit I have later, along with a lot of other cool features I've been working on. Fix: re-add the setPlaying wiring in Stream.vue (set it while launching/playing, clear it on teardown): in case you wanted to just drop in the fix for this, or we can let it slide till the next PR i have around the corner |
|
@LoneAngelFayt two things:
|
|
@gantoine i got you. I can push everything to this branch. I also cleaned up the way the stream page looks, as well as adding some save/state sync since that was opened up since this originally dropped. Need me to prune anything out or are you ok with a larger review all at once? |
useGamepad gates its pad-to-UI translation on the playing store flag, but the v2 Stream player never set it, so button presses kept navigating the app (and a Back/Start press could tear the session down mid-game). Flag the session while launching or playing and clear it on unmount.
The g-prefix sequences and the / search shortcut listened during play, so pressing common game keys (g then h/p/c, or /) navigated away and killed the session. Gate the handler on the playing store flag. EmulatorJS set the flag on play but never cleared it on unmount, leaving pad and hotkey navigation dead after exiting a game. Ruffle never set it at all, so Flash sessions had no protection. Both players now flag the session on play and release it on unmount.
|
got them, it was the global hotkeys that were slipping through (G and H most likely culprits) found ruffle didn't even set playing tag at all so that was just waiting to fail sometime. now they all set playing flag when a game starts and release it on exit, tested end to end. Should be good to go. |
Description
This PR adds a streaming framework to RomM that enables launching games directly from the RomM UI into a native emulator running in a separate container, with full session management, save state control, and volume control, all without leaving the browser.
The emulator runs in a linuxserver Docker container with a Selkies WebRTC stream. A small HTTP broker sidecar inside that container handles game launch, extraction, and I/O. RomM communicates with the broker via a new
/api/streamingbackend endpoint and displays the stream in an embedded player.This initial PR covers four emulators. Additional integrations (PS3/rpcs3, etc.) will follow in separate PRs once the framework is established.
Supported emulators
ps2ngc,wii,wiiuxboxswitchArchitecture
Configuration (
config.yml)Each emulator is its own container entry. Multiple platforms can share a container (e.g.
ngc/wii/wiiuall pointing at the same Dolphin instance) or use separate ones. Thehostis the URL the browser loads the stream from;broker_hostis the server-side URL RomM uses to send commands.Two optional env vars:
BROKER_SECRETauthenticates broker API calls (set on both the RomM container and the emulator container), andSTREAMING_SAVE_TIMEOUTraises the save-and-exit wait for brokers that save slowly (default 45s).Changes
Backend
backend/endpoints/streaming.py- new router: session management, broker proxy, config endpoint. Sessions are claimed by ROM id (the filesystem path is derived server-side from the database, never taken from the client), stored in Redis via atomicSET NX(multi-worker safe), and bound to the claiming user; only the owner or an admin can control or release one.backend/tests/endpoints/test_streaming.py- 19 endpoint tests: auth scopes, claim races, ownership, broker failure paths, force-release.backend/config/config_manager.py-STREAMING_ENABLED+STREAMING_CONTAINERSconfig fieldsbackend/main.py- registers streaming routerFrontend
frontend/src/stores/streaming.ts- Pinia store: config, session lifecycle, per-platform capabilitiesfrontend/src/views/Player/Stream/Player.vue- streaming player view: Selkies iframe, loading overlay, save state controls, volume/mutefrontend/src/components/common/Game/PlayBtn.vue- "Play on [label]" button appears when a streaming container is configured for the platformfrontend/src/layouts/Main.vue- streaming config fetched on app startfrontend/src/plugins/router.ts-rom/:rom/streamrouteConfig
examples/config.example.yml- streaming config example with commentsenv.template-BROKER_SECRETandSTREAMING_SAVE_TIMEOUTvariablesThe player view follows the existing v1 player pattern (same shape as the EmulatorJS/Ruffle views). The route also registers a v2 named view via
v2For, so v2 users get the standard "not ready yet" fallback; a native v2 player is planned as a follow-up once this framework lands.Broker containers
Each emulator has a companion Docker mod repo with the broker sidecar (stdlib-only Python, no extra dependencies):
Each repo includes a
docker-compose.ymlshowing the full setup.UX flow
AI assistance disclosure
This PR was developed with substantial AI assistance (Claude Code). I designed the architecture, made the design decisions, and tested everything against real emulator containers on my own hardware; the majority of the code was written by the AI under my direction and review, and the branch went through multiple AI-assisted review passes before submission.
Checklist
Please check all that apply.
Screenshots (if applicable)