Skip to content

feat: Emulator streaming#3211

Merged
gantoine merged 17 commits into
rommapp:masterfrom
LoneAngelFayt:feat/pcsx2-streaming-v2
Jul 19, 2026
Merged

feat: Emulator streaming#3211
gantoine merged 17 commits into
rommapp:masterfrom
LoneAngelFayt:feat/pcsx2-streaming-v2

Conversation

@LoneAngelFayt

@LoneAngelFayt LoneAngelFayt commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds a streaming framework to RomM that enables launching games directly from the RomM UI into a native emulator running in a separate container, with full session management, save state control, and volume control, all without leaving the browser.

The emulator runs in a linuxserver Docker container with a Selkies WebRTC stream. A small HTTP broker sidecar inside that container handles game launch, extraction, and I/O. RomM communicates with the broker via a new /api/streaming backend endpoint and displays the stream in an embedded player.

This initial PR covers four emulators. Additional integrations (PS3/rpcs3, etc.) will follow in separate PRs once the framework is established.

Supported emulators

Platform slug Emulator Save states Slots Autosave slot Archive extraction
ps2 PCSX2 9 Slot 10 ❌ direct file only
ngc, wii, wiiu Dolphin 7 Slot 8 ❌ direct file only
xbox xemu 9 Slot 10 ❌ direct file only
switch Eden - - ❌ direct file only

Architecture

RomM frontend
  └── StreamingStore (Pinia)
        ├── fetchConfig()       - loads container list from backend on app start
        ├── claimSession()      - POST /api/streaming/sessions
        ├── saveState/loadState - proxied to broker via backend
        ├── saveAndExit()       - save + release session
        └── setVolume/setMute   - PulseAudio control via broker

RomM backend  (/api/streaming/*)
  ├── GET  /config              - returns enabled flag + container list
  ├── POST /sessions            - claims a session, proxies launch to broker
  ├── POST /sessions/{platform}/save-state
  ├── POST /sessions/{platform}/load-state
  ├── POST /sessions/{platform}/save-and-exit
  ├── POST /sessions/{platform}/volume
  ├── POST /sessions/{platform}/mute
  └── DELETE /sessions/{platform} - releases session

Emulator container (linuxserver + broker sidecar)
  ├── POST /launch              - kill current game, launch the requested ROM
  ├── GET  /status              - current session state
  ├── POST /save-state          - hotkey inject, poll for write confirmation
  ├── POST /load-state          - hotkey inject (fire-and-forget)
  ├── POST /save-and-exit
  ├── POST /volume              - pactl set-sink-volume
  └── POST /mute

Configuration (config.yml)

streaming:
  enabled: true
  containers:
    - platform: ps2
      host: https://192.168.1.51:3001       # Selkies WebRTC UI (browser-facing, must be HTTPS)
      broker_host: http://192.168.1.51:8000 # Broker API (server-to-container, HTTP ok)
      label: PCSX2
    - platform: ngc
      host: https://192.168.1.51:3002
      broker_host: http://192.168.1.51:8001
      label: Dolphin

Each emulator is its own container entry. Multiple platforms can share a container (e.g. ngc/wii/wiiu all pointing at the same Dolphin instance) or use separate ones. The host is the URL the browser loads the stream from; broker_host is the server-side URL RomM uses to send commands.

Two optional env vars: BROKER_SECRET authenticates broker API calls (set on both the RomM container and the emulator container), and STREAMING_SAVE_TIMEOUT raises the save-and-exit wait for brokers that save slowly (default 45s).

Changes

Backend

  • backend/endpoints/streaming.py - new router: session management, broker proxy, config endpoint. Sessions are claimed by ROM id (the filesystem path is derived server-side from the database, never taken from the client), stored in Redis via atomic SET NX (multi-worker safe), and bound to the claiming user; only the owner or an admin can control or release one.
  • backend/tests/endpoints/test_streaming.py - 19 endpoint tests: auth scopes, claim races, ownership, broker failure paths, force-release.
  • backend/config/config_manager.py - STREAMING_ENABLED + STREAMING_CONTAINERS config fields
  • backend/main.py - registers streaming router

Frontend

  • frontend/src/stores/streaming.ts - Pinia store: config, session lifecycle, per-platform capabilities
  • frontend/src/views/Player/Stream/Player.vue - streaming player view: Selkies iframe, loading overlay, save state controls, volume/mute
  • frontend/src/components/common/Game/PlayBtn.vue - "Play on [label]" button appears when a streaming container is configured for the platform
  • frontend/src/layouts/Main.vue - streaming config fetched on app start
  • frontend/src/plugins/router.ts - rom/:rom/stream route

Config

  • examples/config.example.yml - streaming config example with comments
  • env.template - BROKER_SECRET and STREAMING_SAVE_TIMEOUT variables

The player view follows the existing v1 player pattern (same shape as the EmulatorJS/Ruffle views). The route also registers a v2 named view via v2For, so v2 users get the standard "not ready yet" fallback; a native v2 player is planned as a follow-up once this framework lands.

Broker containers

Each emulator has a companion Docker mod repo with the broker sidecar (stdlib-only Python, no extra dependencies):

Each repo includes a docker-compose.yml showing the full setup.

UX flow

  1. User clicks Play on PCSX2 (button appears only when a container is configured for that platform)
  2. RomM claims a session; the broker kills any running game and launches the requested ROM
  3. Loading overlay shows a spinner while the emulator starts
  4. Once running: stream fills the player; save state controls and volume appear in the toolbar
  5. Save & Exit saves state then returns to library; session is released for other users

AI assistance disclosure

This PR was developed with substantial AI assistance (Claude Code). I designed the architecture, made the design decisions, and tested everything against real emulator containers on my own hardware; the majority of the code was written by the AI under my direction and review, and the branch went through multiple AI-assisted review passes before submission.

Checklist
Please check all that apply.

  • I've tested the changes locally
  • I've updated relevant comments
  • I've assigned reviewers for this PR
  • I've added unit tests that cover the changes

Screenshots (if applicable)

@LoneAngelFayt

Copy link
Copy Markdown
Contributor Author

here is the broker that goes into the pcsx2 container if you want to take a look at the api inside

https://github.com/LoneAngelFayt/pcsx2-romm-integration

@rommapp rommapp deleted a comment from LoneAngelFayt Apr 7, 2026
@LoneAngelFayt
LoneAngelFayt marked this pull request as ready for review April 10, 2026 03:01
@rommapp rommapp deleted a comment from webysther Apr 11, 2026
@LoneAngelFayt LoneAngelFayt changed the title Feat/emulator streaming Feat: emulator streaming -- streaming framework and pcsx2 integration Apr 11, 2026
@LoneAngelFayt LoneAngelFayt changed the title Feat: emulator streaming -- streaming framework and pcsx2 integration feat: emulator streaming -- streaming framework and pcsx2 integration Apr 12, 2026
@LoneAngelFayt

Copy link
Copy Markdown
Contributor Author

I've finished setting up the integration for dolphin as well, I'm going to split it out into a different PR for each integration

There is thankfully minimal changes but I turn some hardcoding into variables

Once you get done looking through this I'll adapt the others and put them in as I have them completed

@LoneAngelFayt
LoneAngelFayt force-pushed the feat/pcsx2-streaming-v2 branch from 9c8da66 to 6a0e5ff Compare April 28, 2026 14:53
@LoneAngelFayt LoneAngelFayt changed the title feat: emulator streaming -- streaming framework and pcsx2 integration add emulator streaming Apr 28, 2026
@LoneAngelFayt

Copy link
Copy Markdown
Contributor Author

@gantoine I changed my mind, and just added all the emulators I've gotten working along with their respective changes.

I recruited Claude to help make my PR description a little better as well. Should outline everything I've touched and how it all works.

@gantoine gantoine added the on-hold Pending further research or blocked by another issue label Apr 29, 2026
@gantoine
gantoine self-requested a review April 29, 2026 12:33
@gantoine gantoine changed the title add emulator streaming Emulator streaming May 27, 2026
@LoneAngelFayt
LoneAngelFayt force-pushed the feat/pcsx2-streaming-v2 branch from b44b7dc to 6818507 Compare June 13, 2026 16:13
Add /api/streaming endpoints for session claim/release, ROM launch,
save states, volume control, and save-and-exit against per-platform
emulator broker containers. Sessions are stored in Redis with atomic
SET NX claims, owner-bound, and admin force-releasable. Config comes
from config.yml (streaming.enabled + containers list) with a
STREAMING_SAVE_TIMEOUT env override for slow-saving brokers.

Includes 19 endpoint tests covering auth scopes, claim races,
ownership, broker failure paths, and force-release.
Add a Pinia streaming store (config fetch, session lifecycle, broker
controls), a v1 player view at rom/:rom/stream with save state and
volume controls, and a cast button on PlayBtn for platforms with a
configured streaming container. The route registers a v2 named view
via v2For so the v2 fallback screen shows until a v2 player lands.
@LoneAngelFayt
LoneAngelFayt force-pushed the feat/pcsx2-streaming-v2 branch from 6818507 to d94c5b6 Compare July 8, 2026 22:39
@gantoine gantoine removed the on-hold Pending further research or blocked by another issue label Jul 15, 2026
gantoine added 2 commits July 16, 2026 15:46
Move the streaming player UI from v1 to v2 per the v1-frozen rule.
The shared streaming store and backend stay canonical; only the
frontend surface moves.

- New v2 player view (Player/Stream.vue) porting the launch/playing/
  error state machine, session claim, iframe UI auto-hide, volume,
  save/load state, fullscreen, save-and-exit and stop/release.
- Fold streaming into the single Play action via useGameActions:
  routes to /rom/:id/stream when a container is configured, else
  EJS, else Ruffle. Streaming wins over in-browser emulation.
- Hydrate streaming config in AppLayout (v2 users never mount v1
  Main.vue, where v1 put the fetchConfig call).
- Make the rom/:rom/stream route v2-only (v1 default falls back to
  Home).
- Add stream-* i18n keys to all 18 locales.
- Remove the v1 Stream player, PlayBtn cast button and Main.vue
  fetchConfig wiring.

Generated-By: PostHog Code
Task-Id: e091168f-69fd-4d4a-9f12-d473dd3f17c9
@gantoine gantoine changed the title Emulator streaming feat: Emulator streaming Jul 17, 2026
gantoine and others added 8 commits July 16, 2026 21:59
Address review findings on the streaming framework.

Backend (backend/endpoints/streaming.py):
- Session keys now carry a TTL (6h) so an abandoned session (broker
  dead / backend crashed) eventually frees the container instead of
  wedging it until an admin force-releases; control calls refresh the
  TTL so active sessions never expire.
- _container_for_platform validates each entry has a scheme-bearing
  host, skipping malformed ones (mirroring /config) so claim/control
  return a clean 404 instead of a 500 KeyError on container["host"].
- _parse_host_url / _derive_broker_host reject schemeless hosts that
  would produce the broken "//None:8000/..." broker URL and colliding
  session keys; _broker_url raises a clear 502 when no usable host.
- save-and-exit with wait=false drains the session (short-lived key)
  instead of deleting it outright, so a concurrent new claim can't
  /launch on top of a not-yet-dead emulator and lose the in-flight save.
- New tests cover schemeless/missing-host skips, the claim TTL, and
  the wait=false drain window.

Frontend:
- stores/streaming.ts: releaseSession/saveAndExit drop the local
  session record only after the backend confirms, so a failed release
  no longer leaves the user wedged behind their own still-held session.
- v2/views/Player/Stream.vue: disambiguate the 404 from claimSession
  (ROM not found vs container not configured) so the message matches
  the real failure; new stream-error-rom-not-found i18n key in all
  18 locales.

Generated-By: PostHog Code
Task-Id: e091168f-69fd-4d4a-9f12-d473dd3f17c9
Replace English placeholder values for the stream-* keys in play.json
with translations for all 16 non-English locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move raw HTTP calls out of the streaming store into a dedicated
services/api/streaming.ts module, matching the services/api/* convention.
Thin wrappers with no store state (setVolume, setMute, saveState,
loadState) are dropped from the store and called directly via streamingApi
in Stream.vue; the store keeps only state-managing actions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the repeated urllib request construction (URL, secret, payload,
headers, urlopen, parsing) shared by the six broker functions into
_broker_request and its best-effort _broker_request_safe variant. The
best-effort wrappers collapse to a few lines each and the nosec B310
annotation now lives in one place.

Slim the tests with a _streaming() context manager plus _auth/_claim_ok
helpers, removing the repeated cm.get_config patch boilerplate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-platform slot capabilities were duplicated across the frontend store's
hardcoded platformCapabilities() map and the backend's pydantic Field bounds,
independent of the broker's own ceiling. Consolidate the RomM-side knowledge
into one _PLATFORM_CAPABILITIES table in the streaming endpoint:

- /config now ships each container's capabilities, and the frontend selector
  reads them instead of keeping its own copy.
- The SaveState/LoadState request bounds derive from the table, and the routes
  validate the exact per-platform ceiling before calling the broker (clean 422
  instead of a broker 502).

The broker remains the ultimate enforcer; RomM no longer keeps two independent
hardcoded copies of its slot semantics.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gantoine

Copy link
Copy Markdown
Member

@LoneAngelFayt alright i'm happy with the state of this PR, want to give it a spin and see if everything still works?

@LoneAngelFayt

Copy link
Copy Markdown
Contributor Author

Gladly!

@LoneAngelFayt

LoneAngelFayt commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

@gantoine looks like its all working. One thing to note though. Since the v2 migration, it looks like controller inputs are being snagged by the UI as well as the stream. I fix this in a commit I have later, along with a lot of other cool features I've been working on.

Fix: re-add the setPlaying wiring in Stream.vue (set it while launching/playing, clear it on teardown):

import storePlaying from "@/stores/playing";
const playingStore = storePlaying();

const sessionActive = computed(
  () => playerState.value === "playing" || playerState.value === "loading",
);
watch(sessionActive, (active) => playingStore.setPlaying(active));
// and playingStore.setPlaying(false) in onBeforeUnmount

in case you wanted to just drop in the fix for this, or we can let it slide till the next PR i have around the corner

@gantoine

Copy link
Copy Markdown
Member

@LoneAngelFayt two things:

  • i think emulatorjs controls are also being swallowed by the ui navigation, can you try to fix both?
  • i want to land this in 5.1 but we need to release a patch first, so you can either push to this branch or stack a PR

@LoneAngelFayt

Copy link
Copy Markdown
Contributor Author

@gantoine i got you.

I can push everything to this branch.

I also cleaned up the way the stream page looks, as well as adding some save/state sync since that was opened up since this originally dropped. Need me to prune anything out or are you ok with a larger review all at once?

useGamepad gates its pad-to-UI translation on the playing store flag, but
the v2 Stream player never set it, so button presses kept navigating the
app (and a Back/Start press could tear the session down mid-game). Flag
the session while launching or playing and clear it on unmount.
The g-prefix sequences and the / search shortcut listened during play,
so pressing common game keys (g then h/p/c, or /) navigated away and
killed the session. Gate the handler on the playing store flag.

EmulatorJS set the flag on play but never cleared it on unmount, leaving
pad and hotkey navigation dead after exiting a game. Ruffle never set it
at all, so Flash sessions had no protection. Both players now flag the
session on play and release it on unmount.
@LoneAngelFayt

LoneAngelFayt commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

got them, it was the global hotkeys that were slipping through (G and H most likely culprits) found ruffle didn't even set playing tag at all so that was just waiting to fail sometime.

now they all set playing flag when a game starts and release it on exit, tested end to end.

Should be good to go.

@gantoine
gantoine merged commit 62833ca into rommapp:master Jul 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants