[BUG] ggml-vulkan device contexts persist after model unload and degrade system-wide GPU/compositor performance (Windows hybrid Iris Xe + NVIDIA)

[tavsd]

Before You Submit

Please search existing issues to avoid duplicates. Your bug may already be reported! Right now it's just me maintaining this project so many issues can be overwhelming! Help me out by checking first.

I searched for sluggish/lag/GPU/Vulkan issues. Closest are #1755 (Vulkan BSOD on Windows) and #1279 (overlay event-flood, Linux), but neither covers this: system-wide GPU degradation that persists after model unload.

Bug Description

On a hybrid-GPU Windows laptop (Intel Iris Xe drives the display; NVIDIA T500 secondary), the entire PC becomes extremely sluggish while Handy is running — the Intel iGPU sits at 100% with no process showing the GPU usage in Task Manager (only occasional dwm.exe blips). Killing Handy resolves it within seconds. Forcing the transcribe.cpp accelerator to CPU eliminates the problem completely.

Root-cause evidence, in the order we eliminated theories:

1. It is not model residency: model_unload_timeout is set to Immediately and the debug log confirms the model unloads ~100 ms after each transcription — sluggishness persists anyway.
2. It is not the recording overlay: overlay_style is None and the spikes still occur.
3. It is not active work: the debug log shows the native side completely idle (no model loaded, mic stream closed, zero log lines) throughout a multi-minute sluggish period. Whatever burns the GPU is passive state.
4. It is not even inference on the Iris: with Auto, transcribe-cpp binds Vulkan1 = NVIDIA T500 — the Iris never runs compute, yet the Iris is what saturates.
5. CPU backend fixes it: with the accelerator forced to Cpu (parakeet: using cpu backend (strict)), a full session with many transcriptions shows zero sluggishness.

Hypothesis: once the first model load initializes ggml-vulkan, its Vulkan instance/device contexts (created against both GPUs, including the UMA Iris device) persist for the process lifetime — model unload only frees the weights. That residual state degrades DWM composition / video-memory management system-wide. Consistent fingerprints: a freshly launched Handy is fine until the first transcription; the degradation is worst when a GPU-heavy compositing app is foregrounded (deterministically reproducible with a Microsoft Teams incoming screen share focused); the load is unattributable to any process (it shows up as compositor/driver overhead); and process exit — the only thing that destroys the contexts — recovers instantly.

Backend init from the log:

ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Iris(R) Xe Graphics (Intel Corporation) | uma: 1 | fp16: 1 | ...
ggml_vulkan: 1 = NVIDIA T500 (NVIDIA) | uma: 0 | fp16: 1 | ...
load_backend: loaded Vulkan backend from C:\Users\...\AppData\Local\Handy\ggml-vulkan.dll
transcribe_init_backends: 3 compute device(s) registered: Vulkan0, Vulkan1, CPU

Deterministic repro (when it interacts with Teams):

1. Join a Teams meeting with an incoming screen share; GPU ~8–10%. Launch Handy → still fine.
2. Do one transcription in any app (model loads on Vulkan, transcribes, unloads).
3. Focus the Teams window → Intel iGPU 100%, whole desktop sluggish.
4. Exit Handy → back to ~10% within seconds.

It also recurs spontaneously (no Teams focus change) minutes after a transcription, with Handy's log fully idle.

Model: handy-computer/parakeet-unified-en-0.6b-gguf/parakeet-unified-en-0.6b-Q8_0.gguf (streaming). Settings during repro: model_unload_timeout: Immediately, overlay_style: None, always_on_microphone: false, transcribe_accelerator: Auto (bug) / Cpu (workaround), vad_enabled: true.

Workaround: set the transcribe.cpp accelerator to CPU. (Cost on this hardware: streaming Parakeet drops to ~1× real-time, so the post-release wait grows from ~0.3 s to 2–8 s.)

Possibly related: #1755 (Vulkan BSOD on Windows / RTX 5090) — another case of the ggml-vulkan backend misbehaving at the driver level on Windows.

System Information

App Version:

0.9.4

Operating System:

Windows 10 Pro 22H2 (10.0.19045)

CPU:

Intel Core i7-1165G7 (Tiger Lake, 4C/8T) — Lenovo ThinkPad P14s Gen 2i, 48 GB RAM

GPU:

Intel Iris Xe (display, Vulkan0, UMA) + NVIDIA T500 (Vulkan1) — hybrid graphics laptop

Logs

Key excerpts below; full logs available on request.

Sluggish session on Auto (Vulkan): model unloads immediately, log goes idle, system degrades anyway

[12:56:52][transcription][INFO] Live preview finalized in 5.67s model compute for 26.31s streamed audio (4.64x real-time)
[12:56:52][transcription][INFO] Immediately unloading model after streaming transcription
[12:56:52][transcription][DEBUG] Model unloaded manually (took 42ms)
[12:56:52][actions][DEBUG] Text pasted successfully in 226.4027ms
[12:56:52][tray][DEBUG] tray icon change (Idle)
   ... nothing logged for ~3.5 minutes; during this window the Iris hit 100%
       and the desktop became sluggish, with no process showing the GPU usage ...
[13:00:14][transcription][DEBUG] Starting to unload model      <- app being closed by user
[13:00:14][transcription][DEBUG] Model unloaded manually (took 22ms)

Model load during that session: parakeet: using vulkan backend: Vulkan1 (= NVIDIA T500, i.e. compute was NOT on the Iris).

Backend init showing both Vulkan devices registered at startup

[13:04:11][transcribe_cpp][DEBUG] ggml_vulkan: Found 2 Vulkan devices:
[13:04:11][transcribe_cpp][DEBUG] ggml_vulkan: 0 = Intel(R) Iris(R) Xe Graphics (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
[13:04:11][transcribe_cpp][DEBUG] ggml_vulkan: 1 = NVIDIA T500 (NVIDIA) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: none
[13:04:11][transcribe_cpp][INFO] load_backend: loaded Vulkan backend from C:\Users\...\AppData\Local\Handy\ggml-vulkan.dll
[13:04:11][transcribe_cpp][INFO] transcribe_init_backends: 3 compute device(s) registered: Vulkan0, Vulkan1, CPU

Clean session on CPU backend (accelerator = Cpu): multiple transcriptions, zero sluggishness

[13:05:11][transcribe_cpp][INFO] parakeet: using cpu backend (strict)
[13:05:12][transcription][INFO] Loaded whisper model '...parakeet-unified-en-0.6b-Q8_0.gguf' (requested Cpu, bound backend 'CPU', supports_streaming=true)
[13:05:32][transcription][INFO] Live preview finalized in 15.11s model compute for 16.89s streamed audio (1.12x real-time)
[13:05:32][transcription][INFO] Immediately unloading model after streaming transcription
   ... four transcriptions over ~15 minutes, no GPU spike, desktop stayed responsive ...