Move the concept of "internal" data streams into livekit crate from livekit-data-stream - #1304
Open
1egoman wants to merge 3 commits into
Open
Move the concept of "internal" data streams into livekit crate from livekit-data-stream#13041egoman wants to merge 3 commits into
livekit crate from livekit-data-stream#13041egoman wants to merge 3 commits into
Conversation
I think this is going to be a lot cleaner, and mean that other platforms like swift can handle "internal" data streams in their own way
Contributor
Changeset ✓This PR includes a changeset covering all affected packages:
|
1egoman
marked this pull request as ready for review
July 31, 2026 20:41
1egoman
force-pushed
the
data-streams-internal-topics
branch
from
July 31, 2026 20:49
1b9944f to
c57611d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While working on #1286, I realized I made a bit of a mess of the "internal" data stream concept when extracting
livekit-data-streamout into its own crate. This pull request attempts to address this.Currently, what happens today:
livekit-data-stream'sIncomingDataStreamManagertakes a list ofINTERNAL_DATA_STREAM_TOPICS. These are topics used internally for v2 rpc requests / responses and messages associated with these data streams are not exposed to the user. They are filtered in two places:IncomingDataStreamManagerto conditionally expose bothInputEvent::ChunkReceivedandInputEvent::TrailerReceivedfor non internal topicsRoomSessionto expose whichInputEvent::StreamOpenedevents are exposed to the user. Doing this here meant that "internal" data streams could be intercepted at this level and fed into rpc / etc.This is messy - it's a little hard to follow and the filtering is split into these two places / some code duplication. What really made this untenable though was that exposing this internal state over uniffi in #1286 became particularly challenging, since you do want to expose "internal" events downstream in this case - if you didn't, the uniffi consuming client couldn't handle rpc v2 / etc!
What this pull request does: Now,
livekit-data-streamknows nothing about "internal" or "not internal" data streams, and includes the rawtopicof each stream in the associated event. This allows all the filtering to be centralized downstream inRoomSession. This fixes all of these previously mentioned problems and is a lot easier to reason about!