Skip to content

feat(phonic): handle user text input via generate_reply - #3

Draft
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
devin/1781820415-phonic-text-input
Draft

feat(phonic): handle user text input via generate_reply#3
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
devin/1781820415-phonic-text-input

Conversation

@devin-ai-integration

Copy link
Copy Markdown

Summary

The Phonic realtime plugin currently only handles audio input. When a user types text (via LiveKit's chat/text stream), the framework calls update_chat_ctx with the user message followed by generate_reply. Previously, user-role messages in update_chat_ctx were silently ignored — typed input was dropped with no response.

This adds text input support:

# In update_chat_ctx — detect user messages, buffer text:
if isinstance(item, llm.ChatMessage) and item.role == "user":
    self._pending_user_text = item.text_content

# In _send_generate_reply — include buffered text in system_message:
if self._pending_user_text:
    system_message = f'The user sent: "{self._pending_user_text}". Please respond.'
    self._pending_user_text = None

The _pending_user_text bridges the gap between update_chat_ctx (where the framework delivers user text) and _send_generate_reply (where we can pass it to the downstream echo service via the system_message field of generate_reply).

Python counterpart of Phonic-Co/livekit-agents-js#5.

Link to Devin session: https://app.devin.ai/sessions/51df28f166ac4fff8cae41b88dc9bab2
Requested by: @qionghuang6

When a user sends text (e.g. typing in a chat), the framework calls
update_chat_ctx with the user message followed by generate_reply.
Previously the Phonic plugin ignored user messages in update_chat_ctx,
so text input was silently dropped.

This change:
- Detects new user messages in update_chat_ctx and stores the text
- Includes the user text in the generate_reply system_message sent
  to the Phonic downstream service, so the LLM can see and respond
  to typed user input

Co-Authored-By: qiong@phonic.co <qionghuang6@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

qionghuang6 and others added 3 commits June 18, 2026 18:13
say() drives speech directly and supersedes any buffered user text turn,
so drop _pending_user_text to avoid it leaking into a later generate_reply.
Addresses the cancellation edge case flagged in review.

Co-authored-by: Cursor <cursoragent@cursor.com>
…onic-text-input

Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	livekit-plugins/livekit-plugins-phonic/livekit/plugins/phonic/realtime/realtime_model.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant