Skip to content

api

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

api

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
routers
routers
 
 
segments_engine
segments_engine
 
 
websockets
websockets
 
 
.env.sample
.env.sample
 
 
README.md
README.md
 
 
__init__.py
__init__.py
 
 
config.py
config.py
 
 
main.py
main.py
 
 
utils.py
utils.py
 
 

README.md

API Notes

This folder contains the FastAPI backend for VRChat Media Gateway.

The backend is responsible for:

  • media download and conversion for YouTube, SoundCloud, Telegram photos/videos/stickers/music tracks/voice messages, generic images/audio/videos, animated GIF motion media, and local media
  • websocket RPC for Spotify Desktop control
  • live HLS segment generation for Spotify
  • tunnel URL discovery from logs/cloudflared.log

Runtime Model

  • FastAPI listens on 127.0.0.1:5000
  • nginx on 127.0.0.1:8080 proxies /api/* and /api/ws/* to FastAPI
  • generated HLS output is written to ../html/streams/
  • /local-api/* stays on loopback-only FastAPI and is intentionally outside the nginx/tunnel path

For full-stack usage, start the project from the repository root with:

.\run stream server.bat

Main Endpoint Groups

  • /api/stream-yt
  • /api/stream-sc
  • /api/stream-image
  • /api/stream-audio
  • /api/stream-video
  • /api/stream-tg-media
  • /api/stream-tg-image
  • /api/stream-tg-video
  • /api/stream-tg-audio
  • /api/tg-post-info
  • /api/stream-spotify
  • /api/stream-spotify-clear
  • /api/tunnel
  • /api/ws/spotify
  • /local-api/stream-local-path-build-start
  • /local-api/stream-local-upload-build-start
  • /local-api/stream-local-build-status
  • /local-api/clear-cache-all

Animated GIF behavior:

  • generic .gif sources are treated as motion/video instead of the still-image path
  • Telegram animated GIF posts are classified as video-like media and converted through the Telegram video pipeline

Telegram sticker behavior:

  • static Telegram stickers such as .webp are classified as image media and converted through the Telegram image path
  • video/webm Telegram stickers are classified as video media and converted through the Telegram video path
  • animated .tgs Telegram stickers are rendered through rlottie_python, converted to motion media, and then sent through the Telegram video path

Telegram audio behavior:

  • Telegram music documents and Telegram voice messages are classified as audio media and converted through the Telegram audio path
  • /api/stream-tg-audio is the direct Telegram audio endpoint
  • /api/stream-tg-media auto-detects Telegram audio posts and routes them to the same audio pipeline
  • mono Telegram voice messages are preserved correctly when converted to stereo AAC HLS output
  • Telegram music tracks build poster-style audio HLS with cover/title/performer when Telegram exposes that metadata

Audio poster behavior:

  • /api/stream-sc now builds poster-style audio HLS with SoundCloud artwork, title, performer, and duration when yt-dlp exposes those fields
  • /api/stream-audio builds poster-style audio HLS and uses embedded tags/cover art when the remote audio source exposes them
  • local audio builds through /local-api/stream-local-path-build-start and /local-api/stream-local-upload-build-start also use embedded tags/cover art when available

Telegram post-text behavior:

  • /api/stream-tg-media, /api/stream-tg-image, and /api/stream-tg-video accept optional with_text=1
  • when with_text=1, Telegram post text is rendered on a black panel below the media instead of over the media
  • text-only Telegram posts can also build a text-only HLS output through /api/stream-tg-media
  • /api/tg-post-info is a lightweight metadata probe used by the browser extension to detect whether a Telegram post is text-only before starting a managed build

Non-Spotify HLS tuning:

  • /api/stream-yt
  • /api/stream-sc
  • /api/stream-image
  • /api/stream-audio
  • /api/stream-video
  • /api/stream-tg-media
  • /api/stream-tg-image
  • /api/stream-tg-video
  • /api/stream-tg-audio
  • /local-api/stream-local-path-build-start
  • /local-api/stream-local-upload-build-start

Those routes accept optional segment_time=<seconds>. If the param is missing, the backend uses the default from HLS_OPTS in config.py. The browser extension popup now stores that value in its own Streaming settings section and sends it for non-Spotify exports.

Python dependency note:

  • rlottie_python is required for animated Telegram .tgs sticker export

Telegram Config

The backend loads api/.env first and falls back to the root environment if that file is missing.

See api/.env.sample for the Telegram variables:

  • TG_API_ID
  • TG_API_HASH
  • TG_PASSWORD
  • TG_SESSION

Generate a session from the repository root with:

python auxillary/get_tg_session.py

Spotify Note

Spotify features depend on the in-memory websocket registry in this backend. Use the single-process full-stack path when you need Spotify Desktop or Spotify Web export flows.

/api/stream-spotify accepts optional Spotify HLS tuning through query params:

  • segment_time=<seconds>
  • prefetch=<count>
  • show_info=<0|1>

Example:

/api/stream-spotify?url=https://open.spotify.com/track/...&segment_time=2&prefetch=10&show_info=1

If those params are missing, the backend uses the defaults from SPOTIFY_HLS_OPTS in config.py.

The Spicetify bridge stores those values in its Streaming settings section and sends them through the VRChat button and the Spotify cache-clear flow so different Spotify streaming presets do not share the same cache folder.

When show_info=1, the backend asks the Spicetify bridge for current track metadata, renders a poster frame with cover/title/artist, and muxes that poster video together with the live Spotify audio capture. When show_info=0, Spotify stays on the older audio-only live HLS path.

Local Media Note

Local media ingestion is split out from the public API on purpose.

  • /local-api/stream-local-path-build-start accepts an absolute local filesystem path and only works when FastAPI is running on the same machine that can read that file
  • /local-api/stream-local-upload-build-start accepts raw uploaded file bytes for image, video, and audio media
  • /local-api/stream-local-build-status polls local build jobs
  • /local-api/clear-cache-all stops active stream writers, resets build job state, and wipes generated cache directories on the local machine

Security rules for that path:

  • local media routes reject non-loopback callers
  • they are not intended to be proxied by nginx or exposed through Cloudflare Tunnel
  • browser file pickers and drag-and-drop prefer filesystem handles when Chromium exposes them, and fall back to upload otherwise

Large local upload note:

  • the backend upload ceiling is controlled by LOCAL_UPLOAD_MAX_BYTES in config.py
  • the default is 256 GiB, which is intended for very large local media uploads on the same machine

The final playback URL still comes from /streams/<sid>/index.m3u8, but the local file ingestion step itself is local-only.