feat(llm): keep provider-specific handlers attached across realtime fallback swaps - #6628
Open
biztex wants to merge 1 commit into
Open
feat(llm): keep provider-specific handlers attached across realtime fallback swaps#6628biztex wants to merge 1 commit into
biztex wants to merge 1 commit into
Conversation
…allback swaps Plugin-specific realtime events (e.g. the openai plugin''s openai_server_event_received) can only be observed on the child session, but a failover closes that child and brings up a new one - silently dropping every handler the application attached. Users had to isinstance-check the private _FallbackRealtimeSession and resubscribe by hand, with no signal for when a swap happened. Add a handler registry to the fallback session: - on_active_session(event, callback): attaches to the current child and re-attaches automatically to every child a failover or restart brings up. Events a child''s provider never emits simply don''t fire, so handlers stay inert while a different provider''s model is active. - off_active_session(event, callback): detaches everywhere. - active_session property: the child currently in use, documented as replaced on swap. Fixes livekit#6556
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6556. Related: #6553 (reduces the need to isinstance-check the private class at all).
Problem
Plugin-specific realtime events (e.g. the openai plugin's
openai_server_event_received/openai_client_event_queued) can only be observed on the child session ofRealtimeModelFallbackAdapter— the wrapper only forwards the generic event set. When a failover swaps the child, every handler the application attached to it is silently lost. Today the workaround is isinstance-checking the private_FallbackRealtimeSession, reaching into._active, and manually resubscribing — with no signal for when a swap happened (the issue's exact pain).Change
A handler registry on the fallback session, so resubscription is automatic:
on_active_session(event, callback)— attaches to the current child and re-attaches to every child a failover or restart brings up (_bind/_unbindalready run on each swap; the registry rides along). Events a child's provider never emits simply don't fire, so openai-specific handlers are inert while a different provider's model is active — exactly the "resubscribe when fallbacked to the same plugin-type model" semantics requested, with no provider bookkeeping.off_active_session(event, callback)— detaches everywhere.active_sessionproperty — the child currently in use, documented as replaced on swap (mirrors the naming already used by the test fakes).Tests
Four new cases in
tests/test_realtime_fallback.pyon the existing fake-model harness: the property exposes the live child; a registered handler receives events from both the pre-swap and post-swap child;off_active_sessiondetaches across swaps; and the direct-attachment failure mode is documented by contrast.Happy to adjust naming/shape —
on_active_sessionwas chosen to make the "re-attached on swap" semantics part of the name.