Skip to content

Move the concept of "internal" data streams into livekit crate from livekit-data-stream - #1304

Open
1egoman wants to merge 3 commits into
mainfrom
data-streams-internal-topics
Open

Move the concept of "internal" data streams into livekit crate from livekit-data-stream#1304
1egoman wants to merge 3 commits into
mainfrom
data-streams-internal-topics

Conversation

@1egoman

@1egoman 1egoman commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

While working on #1286, I realized I made a bit of a mess of the "internal" data stream concept when extracting livekit-data-stream out into its own crate. This pull request attempts to address this.

Currently, what happens today: livekit-data-stream's IncomingDataStreamManager takes a list of INTERNAL_DATA_STREAM_TOPICS. These are topics used internally for v2 rpc requests / responses and messages associated with these data streams are not exposed to the user. They are filtered in two places:

  • Within IncomingDataStreamManager to conditionally expose both InputEvent::ChunkReceived and InputEvent::TrailerReceived for non internal topics
  • Within RoomSession to expose which InputEvent::StreamOpened events are exposed to the user. Doing this here meant that "internal" data streams could be intercepted at this level and fed into rpc / etc.

This is messy - it's a little hard to follow and the filtering is split into these two places / some code duplication. What really made this untenable though was that exposing this internal state over uniffi in #1286 became particularly challenging, since you do want to expose "internal" events downstream in this case - if you didn't, the uniffi consuming client couldn't handle rpc v2 / etc!

What this pull request does: Now, livekit-data-stream knows nothing about "internal" or "not internal" data streams, and includes the raw topic of each stream in the associated event. This allows all the filtering to be centralized downstream in RoomSession. This fixes all of these previously mentioned problems and is a lot easier to reason about!

1egoman added 2 commits July 31, 2026 16:21
I think this is going to be a lot cleaner, and mean that other platforms
like swift can handle "internal" data streams in their own way
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Changeset ✓

This PR includes a changeset covering all affected packages:

Package Bump
livekit patch
livekit-data-stream patch
livekit-ffi patch

@1egoman
1egoman marked this pull request as ready for review July 31, 2026 20:41
@1egoman
1egoman requested a review from ladvoc as a code owner July 31, 2026 20:41
devin-ai-integration[bot]

This comment was marked as resolved.

@1egoman
1egoman requested a review from xianshijing-lk July 31, 2026 20:46
@1egoman
1egoman force-pushed the data-streams-internal-topics branch from 1b9944f to c57611d Compare July 31, 2026 20:49
devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant