Skip to content

Fix IME commitText() to enable voice dictation and swipe typing#1556

Open
AFK-Automations wants to merge 1 commit intomoonlight-stream:masterfrom
AFK-Automations:fix/ime-committext
Open

Fix IME commitText() to enable voice dictation and swipe typing#1556
AFK-Automations wants to merge 1 commit intomoonlight-stream:masterfrom
AFK-Automations:fix/ime-committext

Conversation

@AFK-Automations
Copy link
Copy Markdown

Problem

Android IME text input via commitText() is silently dropped during streaming. This means Gboard voice dictation, swipe typing, and autocomplete suggestions never reach the host - the keyboard appears to listen but no text arrives on the remote PC. Reported in #1496.

Root Cause

Moonlight handles keyboard input through onKeyDown()/onKeyUp() which only processes individual KeyEvent objects. When Gboard voice engine or swipe/autocomplete completes, Android calls commitText() on the active InputConnection. No custom InputConnection was provided, so the committed text was never forwarded to the host.

Fix

  • Created StreamInputConnection extending BaseInputConnection that intercepts commitText() and routes text through the existing sendUtf8Text() protocol method
  • Added onCreateInputConnection() override on StreamView to provide the custom InputConnection during streaming
  • Also handles deleteSurroundingText() for IME-driven backspace/delete
  • setComposingText() and finishComposingText() handled gracefully to prevent errors during in-progress dictation

What works after this change

  • Gboard voice dictation -> text appears on host
  • Gboard swipe typing -> text appears on host
  • Autocomplete/suggestion taps -> text appears on host
  • Regular key-by-key typing -> unchanged, still works through existing KeyEvent path
  • Combining accents -> unchanged, still works

Files changed

  • StreamInputConnection.java -- new file
  • StreamView.java -- added onCreateInputConnection() override
  • Game.java -- passes NvConnection reference to StreamView

Tested on Android with Gboard, connected to Sunshine host on Windows 11. Voice dictation, swipe typing, autocomplete, and regular typing all function correctly. No regressions observed.

Closes #1496

Moonlight receives keyboard input via KeyEvent (onKeyDown/onKeyUp) but
does not handle IME commitText() events. This means Gboard voice
dictation, swipe typing, and autocomplete suggestions are silently
dropped.

Adds StreamInputConnection (extending BaseInputConnection) that
intercepts commitText() and routes text through the existing
sendUtf8Text() protocol method. Also handles deleteSurroundingText()
for IME-driven backspace/delete operations.

Closes moonlight-stream#1496
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature request]: Add support for using Android soft keyboard as virtual input device / 支持使用 Android 虚拟键盘作为输入设备

1 participant