Skip to content

feat: add WASAPI loopback capture mixed with microphone (Windows)#1259

Open
MoulinLouis wants to merge 1 commit intocjpais:mainfrom
MoulinLouis:feat/system-audio-loopback
Open

feat: add WASAPI loopback capture mixed with microphone (Windows)#1259
MoulinLouis wants to merge 1 commit intocjpais:mainfrom
MoulinLouis:feat/system-audio-loopback

Conversation

@MoulinLouis
Copy link
Copy Markdown

@MoulinLouis MoulinLouis commented Apr 9, 2026

Before Submitting This PR

Human Written Description

I do a lot of technical interviews over Google Meet and I've been wanting to transcribe both sides of the conversation, not just my mic. Right now I have to use a separate app for that and it's annoying. Since cpal already supports WASAPI loopback natively, I figured I'd try adding it to Handy.

I saw #248 and @cjpais's response about being open to a well-written PR. I also know about the feature freeze, so totally fine if this gets rejected.

I tried to keep it as non-intrusive as possible:

  • It's Windows-only, uses WASAPI loopback which cpal already supports, so no new deps
  • It mixes system audio with the mic instead of replacing it, so you still get both sides
  • Off by default, and when it's off the code path is exactly the same as before
  • One setting, two commands, one small toggle at the bottom of the Sound section

Related Issues/Discussions

Related to #248, same feature request but with a mic+loopback mixing approach instead of switching between sources.

Community Feedback

A few people asked for this in #248 for similar use cases (transcribing meetings, calls, etc). It's definitely a niche feature though, which is why I kept it as a simple opt-in toggle.

Testing

  • Toggling "Record System Audio" on while playing a YouTube video: transcription captures the video audio mixed with mic input
  • Toggling it off: normal microphone-only transcription works exactly as before
  • Changing the output device while loopback is active restarts the stream correctly
  • Mute-while-recording is correctly disabled when loopback is on (muting the output device would silence the capture source)
  • Audio feedback (start/stop beeps) still plays in loopback mode
  • Always-on microphone mode works correctly with loopback enabled
  • Verified on Windows 11 with WASAPI

Changes overview

Backend (Rust):

  • settings.rs: new record_system_audio: bool field
  • recorder.rs: open() accepts optional loopback device, run_consumer() resamples both streams to 16kHz and mixes them before VAD
  • audio.rs: always resolves mic device, optionally resolves output device for loopback
  • commands/audio.rs: set_record_system_audio / get_record_system_audio commands
  • actions.rs: mute skipped when loopback active (output device is the capture source)

Frontend (React/TypeScript):

  • RecordSystemAudio.tsx: toggle component following existing patterns
  • GeneralSettings.tsx: toggle at bottom of Sound section, MuteWhileRecording disabled when active
  • settingsStore.ts: updater and normalization
  • i18n translations added to all 20 locales

Screenshots/Videos (if applicable)

N/A, simple toggle in existing Sound settings section

AI Assistance

  • AI was used (please describe below)

If AI was used:

  • Tools used: Claude Code (claude-opus-4-6)
  • How extensively: AI did most of the implementation, I handled the design decisions and reviewed everything

Add a "Record System Audio" toggle (Windows only) that captures
WASAPI loopback audio from the selected output device and mixes it
with the microphone input before transcription.

Both streams are resampled to 16kHz independently and mixed sample-
by-sample before VAD processing, so speech from either source is
captured. Mute-while-recording is skipped when loopback is active
since muting the output device would silence the capture source.

Backend:
- New record_system_audio bool setting with get/set Tauri commands
- AudioRecorder.open() accepts optional loopback_device for mixing
- run_consumer() handles dual-stream resampling and mixing
- AudioRecordingManager always opens mic, optionally adds loopback
- set_selected_output_device restarts stream when loopback active
- Mute skipped in loopback mode, audio feedback still plays

Frontend:
- RecordSystemAudio toggle component (Windows only)
- Settings store updater and normalization
- MuteWhileRecording disabled when loopback active
- Output device selector stays enabled for loopback source selection
- i18n keys for the new toggle
@cjpais
Copy link
Copy Markdown
Owner

cjpais commented Apr 9, 2026

Can you send screenshots, I'm open to this as a lot of people ask and if it's well implemented we can pull it in.

@MoulinLouis
Copy link
Copy Markdown
Author

handy_sound_1 handy_sound_2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants