Skip to content

TODO.md

Latest commit

 

History

History
86 lines (65 loc) · 6.75 KB

TODO.md

File metadata and controls

86 lines (65 loc) · 6.75 KB
Raw

NEXT

  • otoji-tray: replace text title with SF Symbol mic icon
  • otoji listen: idle CPU gate before RNNoise (76% → ~0%)
  • macOS: "Launch at Login" tray toggle — rs/adapters/macos/src/launch_at_login.rs (writes/removes ~/Library/LaunchAgents/com.snomiao.capslockx.plist; enable only registers for next login, disable also launchctl bootouts), checkbox item wired in tray.rs/prefs.rs. Runs dist/CapsLockX-dev.app/Contents/MacOS/clx -f (branded .app identity for the Accessibility list) with ORT_DYLIB_PATH set for ten-vad's dlopen; deliberately no KeepAlive (would fight clx's startup dedup).
  • Windows: "Launch at Login" tray toggle (mirror of the macOS feature above)
    • Add a CheckMenuItemBuilder-based checkbox item to the tray menu built in rs/adapters/windows/src/main.rs (alongside the existing prefs/config_dir/quit items around line 234-246), wired via on_menu_event.
    • Backing store: a new module (e.g. rs/adapters/windows/src/launch_at_login.rs) that writes/removes a value under the per-user registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Run (name CapsLockX, value = path to the exe, quoted). This is the standard lightweight mechanism for per-user autostart — no admin rights, no Task Scheduler XML. Use the windows crate (already a dependency here) via RegSetValueExW/RegDeleteValueW on HKEY_CURRENT_USER, or pull in the auto-launch crate if it simplifies things.
    • is_enabled() = registry value exists and points at the current exe path (like the mac version, treat the registry as the single source of truth — no separate config flag to desync).
    • Note: unlike macOS, Windows Run-key autostart just launches the exe directly at login — there's no separate watchdog/crash-restart wrapper on this platform to route through, so Program = current exe path with no wrapper args.
  • otoji-tray: extract objc-helpers.rs shared with CLX tray.rs (~150 lines duplicated)
  • CLX↔otoji-tray state channel: drop CLX's own tray, otoji-tray reflects CLX mode/PTT via state file

发展路线 🛰️ RoadMap

CapsLockX 的核心理念是：简化系统操作逻辑，提升操作效率，且不与原有习惯键位冲突。

  1. 按 CapsLockX + - 键显示对应帮助（目前的显示样式相当草率）
  2. i18n （eh 这个真得有）
  3. 自动更新（虽然 git pull 一下也不是不行）
  4. 初次使用上手教程（这个现在有点简陋……）
  5. 插件管理器（虽然文件系统也可以搞定）
  6. 自动配置同步功能（虽然一般来说扔 onedrive 就够）
  7. 易用的选项配置的 UI 界面（虽然改 ini 也不是什么难事）
  8. 执行外部代码（Python、Nodejs、外部 AHK、Bash、……）（虽然写个脚本 run 一下也并不算麻烦）

Rust Port – Active Investigation

BUG: Shift+HJKL Text Selection (IN PROGRESS)

Problem: CapsLock+Shift+HJKL should select text (Shift+Arrow) but only moves cursor.

Root Cause: AccModel callback fires on background thread (clx-acc-ticker). SendInput from this thread causes the OS to insert phantom modifier key-up events between injected events, cancelling the Shift state before Arrow keys arrive.

Approaches tried & results:

# Approach Result
1 Plain key_tap(Arrow) relying on held Shift No selection — phantom Shift UP from OS cancels Shift before Arrow arrives
2 Separate key_down(Shift) + key_tap(Arrow) Phantom Shift UP between separate SendInput calls
3 Atomic key_tap_n_with_mod (single SendInput batch) Phantoms still within batch; works on 2nd+ ticks but not 1st
4 Suppress phantom Shift UP in hook (GetAsyncKeyState) Fixes selection but causes stuck Shift — our injected Shift keeps GetAsyncKeyState returning "down", blocking real releases
5 GetAsyncKeyState for shift detection in cursor_action Correctly detects shift, doesn't fix OS-level Shift state for arrows
6 Skip modifier injection if already held (GetAsyncKeyState check in key_tap_n_with_mod) Phantom UP still appears after AHK's own Shift injection — not caused by our code

Key finding: The phantom Shift UP (inj=false extra=0x0) appears immediately after ANY injected Shift DN — whether from our code or AHK. This is an OS-level behavior, possibly related to how WH_KEYBOARD_LL hooks interact with injected modifier keys. The phantom cancels the Shift state at the OS level before our Arrow key arrives.

Next steps:

  • Try PostMessage/SendMessage WM_KEYDOWN to target window with shift flag in lParam (bypasses SendInput entirely)
  • Try scan-code based injection (KEYEVENTF_SCANCODE flag)
  • Try injecting on the hook thread (not the AccModel ticker thread) — use a message queue to marshal calls
  • Check if the AHK version has the same phantom issue (it works, so how does it avoid it?)

E2E test infra: rs/test-shift-select.ahk creates its own AHK Gui Edit control (no Notepad dependency)

E2E test results (5/8 pass):

  • PASS: CLX+L, CLX+H, CLX+K, CLX+J (plain cursor movement), Shift-not-stuck
  • FAIL: CLX+Shift+L, Shift+CLX+L, CLX+Shift+H (selection)

DONE: CycleWindows cross-monitor fix

  • AHK + Rust: windows cycle within monitor -> next monitor -> virtual desktop switch
  • get_app_windows() sorts by (monitor_index, hwnd) for stable monitor-grouped ordering

DONE: AHK modules opt-in

  • --with-ahk flag required to spawn AHK module loader (default: Rust-only)

DONE: Admin elevation config

  • request_admin: true in %APPDATA%\CapsLockX\config.json triggers UAC elevation on startup

FEAT: Voice Input (CLX+V) — Streaming Speech-to-Text

Plan: plan/voice-input/README.md | plan/voice-input/TODO.md

  • CLX+V click = toggle continuous listening; CLX+V hold = listen while held
  • VAD splits audio into utterances, sends chunks to server incrementally
  • 3-stage streaming pipeline: local Whisper (fast draft) → server Whisper (refined) → LLM typo-fix (polished)
  • Text typed at cursor, each stage replaces previous in-place
  • Server: new endpoint on brainstorm.snomiao.com combining Whisper + LLM
  • Client: cpal for cross-platform audio, webrtc-vad for voice detection
  • Phases: hotkey → audio capture → VAD → server endpoint → HTTP client → local Whisper (optional)

BUG: macOS mouse drag doesn't move windows in realtime

Problem: Space+E (left click hold) + WASD to drag a window only moves the window when E is released, instead of moving it continuously like a trackpad drag. Cause: mouse_move sends CGEventType::MouseMoved but while a button is held, macOS expects CGEventType::LeftMouseDragged events instead. Need to track button state and switch event type accordingly.