Merge pull request #27 from decibri/development

feat: stop recording when the speaker stops talking

Author: rossarmstrong

CHANGELOG.md

@@ -7,6 +7,67 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.5.0] - 2026-07-15
+
+### Added
+
+- Silence-stopped capture. `capture_audio` gains `stop_on_silence`
+  (default `false`): when set, the recording ends when the speaker stops
+  talking, detected with the Silero voice-activity model that ships
+  inside decibri and runs on-device through its bundled ONNX Runtime, so
+  nothing extra is downloaded and no audio leaves the machine. In that
+  mode `duration_ms` becomes a maximum rather than an exact length, and
+  the new `silence_ms` argument (default 1000, range 100-10000) sets how
+  much continuous silence ends the recording. Recording ends at the
+  first of: `silence_ms` of continuous post-speech silence, the
+  `duration_ms` ceiling, or 10 seconds with no speech at all. The result
+  reports the actual captured `duration_ms`, which condition ended it
+  (`stopped_by`), and whether speech was heard (`speech_detected`).
+  Every stop condition is measured in delivered audio, never wall-clock
+  timers, so the stop point is an exact function of the audio; the
+  detector only decides when to stop and never alters the recorded
+  samples, and a fixed-duration capture is unchanged byte for byte.
+- `voice_query` stops on silence by default, with a 15 second ceiling.
+  Its only consumer is speech transcription and a spoken query has no
+  known length, so the previous fixed window truncated long questions
+  and recorded silence after short ones. `stop_on_silence: false`
+  restores the previous fixed 5 second window exactly.
+- Deterministic test suites, runnable on every platform with no
+  microphone, whisper, or Ollama: the silence-stop state machine
+  (scripted per-chunk scores pinning the exact stop chunk and byte count
+  for each stop condition) and the `voice_query` no-answer contract
+  (including that a filler-only transcription never reaches the model).
+
+### Changed
+
+- `voice_query`'s no-answer results are restructured, and a caller that
+  branched on the previous shapes will observe the change. No speech at
+  all, and speech that produced no usable words, are now non-error
+  results whose machine-readable contract is `speech_detected` (`false`
+  vs `true`) with `transcription: null`, so a caller distinguishes "the
+  user was silent" from "the user spoke but produced no words" without
+  parsing prose. Empty transcription previously returned `isError: true`
+  and no longer does. A transcription failure, an unreachable or erroring
+  Ollama, and an empty model response remain `isError: true`, each
+  surfacing its real cause. The rule is that a pipeline that ran and
+  found no words is a success, while a broken dependency is an error.
+  The structured fields are the contract; any human-readable message is
+  not, so callers branch on the fields rather than the prose.
+
+### Fixed
+
+- Whisper's non-speech markers no longer reach the language model. When
+  a recording contained silence or noise, whisper returns markers such
+  as `[BLANK_AUDIO]`, `[MUSIC]`, or `(silence)` as ordinary text, and
+  these were forwarded to Ollama as the user's query, so the model
+  answered a question the user never asked. A transcription is now
+  treated as having usable words only if, after removing every bracketed
+  and parenthesised marker, at least one letter or digit remains, so a
+  marker-only or whitespace-only transcription is reported as no words
+  and never reaches the model. The check is conservative: a real
+  transcription that merely contains a bracketed word keeps its words
+  and is unaffected.
+
 ## [0.4.0] - 2026-07-14
 
 ### Fixed
@@ -251,7 +312,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   listing, WAV output validation, and error responses.
 - Tag-triggered npm publish workflow.
 
-[Unreleased]: https://github.com/decibri/mcp-listen/compare/v0.4.0...HEAD
+[Unreleased]: https://github.com/decibri/mcp-listen/compare/v0.5.0...HEAD
+[0.5.0]: https://github.com/decibri/mcp-listen/compare/v0.4.0...v0.5.0
 [0.4.0]: https://github.com/decibri/mcp-listen/compare/v0.3.0...v0.4.0
 [0.3.0]: https://github.com/decibri/mcp-listen/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/decibri/mcp-listen/compare/v0.2.0...v0.2.1

README.md

@@ -91,7 +91,7 @@ Intel Mac (darwin-x64) is not supported: Apple has discontinued the platform and
 
 ## Tool Reference
 
-Arguments are validated before anything is recorded or written. An argument a tool does not declare is rejected with an error naming it, rather than silently ignored. `duration_ms` must be an integer between 100 and 30000; `device` must be a non-negative integer index or a non-empty string id from `list_audio_devices`. A rejected call writes nothing to disk.
+Arguments are validated before anything is recorded or written. An argument a tool does not declare is rejected with an error naming it, rather than silently ignored. `duration_ms` must be an integer between 100 and 30000; `silence_ms` an integer between 100 and 10000; `stop_on_silence` a boolean; `device` a non-negative integer index or a non-empty string id from `list_audio_devices`. A `silence_ms` that cannot take effect (passed without silence-stopping active) is rejected for the same reason unknown arguments are. A rejected call writes nothing to disk.
 
 ### list_audio_devices
 
@@ -112,16 +112,18 @@ The `id` is stable across reboots and device changes. The `index` is positional
 
 ### capture_audio
 
-Records audio from the microphone and saves as a WAV file.
+Records audio from the microphone and saves as a WAV file. Records for exactly `duration_ms` by default, or until the speaker stops talking with `stop_on_silence: true`.
 
 **Parameters:**
 
 | Parameter | Type | Default | Description |
 | ---------- | ------ | --------- | ------------- |
-| `duration_ms` | number | 5000 | Recording duration in milliseconds (100-30000) |
+| `duration_ms` | number | 5000 | Recording duration in milliseconds (100-30000). A maximum, not an exact length, when `stop_on_silence` is true |
 | `device` | number or string | system default | Device index or stable device `id` from `list_audio_devices` |
+| `stop_on_silence` | boolean | false | Stop when the speaker stops talking, detected with on-device voice activity detection (Silero VAD, bundled, no download) |
+| `silence_ms` | number | 1000 | Continuous silence in milliseconds that ends a `stop_on_silence` recording (100-10000). Requires `stop_on_silence: true` |
 
-**Example response:**
+**Example response (fixed duration):**
 
 ```json
 {
@@ -133,16 +135,34 @@ Records audio from the microphone and saves as a WAV file.
 }
 ```
 
+**Example response (`stop_on_silence: true`):**
+
+```json
+{
+  "path": "/tmp/mcp-listen-1712345678901.wav",
+  "duration_ms": 2600,
+  "sample_rate": 16000,
+  "channels": 1,
+  "size_bytes": 83244,
+  "stopped_by": "silence",
+  "speech_detected": true
+}
+```
+
+With `stop_on_silence`, `duration_ms` in the response is the actual captured length, and `stopped_by` says how the recording ended: `"silence"` (the speaker finished), `"ceiling"` (the `duration_ms` maximum was reached), or `"no_speech_timeout"` (nobody spoke for 10 seconds; the WAV is still returned, with `speech_detected: false`, so silence is a reported outcome rather than an error). Detection runs per ~100ms audio buffer, so the effective hangover rounds up to the next buffer, and the recording keeps everything from the start of the call through the stop decision: nothing is gated or clipped at speech boundaries, and the audio itself is byte-identical to a fixed-duration capture of the same sounds.
+
 ### voice_query
 
-Full voice pipeline: capture audio, transcribe with whisper.cpp, send to Ollama, return the response. Entirely offline.
+Full voice pipeline: capture audio, transcribe with whisper.cpp, send to Ollama, return the response. Entirely offline. Recording stops automatically when the speaker stops talking; pass `stop_on_silence: false` for a fixed-length recording.
 
 **Parameters:**
 
 | Parameter | Type | Default | Description |
 | ----------- | ------ | --------- | ------------- |
-| `duration_ms` | number | 5000 | Recording duration in milliseconds (100-30000) |
+| `duration_ms` | number | 15000 / 5000 | Maximum recording duration in milliseconds (100-30000). Default 15000 while silence-stopping is active, 5000 with `stop_on_silence: false` |
 | `device` | number or string | system default | Device index or stable device `id` from `list_audio_devices` |
+| `stop_on_silence` | boolean | true | Stop recording when the speaker stops talking. Pass false to record for exactly `duration_ms` |
+| `silence_ms` | number | 1000 | Continuous silence in milliseconds that ends the recording (100-10000) |
 | `whisper_model` | string | ggml-base.en.bin | Path or filename of Whisper GGML model |
 | `language` | string | en | Language code for transcription |
 | `model` | string | llama3.2 | Ollama model name |
@@ -158,12 +178,52 @@ Full voice pipeline: capture audio, transcribe with whisper.cpp, send to Ollama,
 }
 ```
 
+**Result outcomes.** `voice_query` reports five distinct outcomes. The **structured fields are the contract** (`isError`, `speech_detected`, `transcription`); any `message` is a human-readable hint whose wording is not part of the contract, so a caller branches on the fields, never on the prose. The rule is simple: **if the pipeline ran, the result is a success (even when it found no words); if a dependency broke, the result is an error.**
+
+| Outcome | `isError` | `speech_detected` | `transcription` | `response` |
+| --- | --- | --- | --- | --- |
+| Normal | absent | (`true`/omitted) | the text | the answer |
+| No speech at all | absent | `false` | `null` | `null` |
+| Speech, but no transcribable words | absent | `true` | `null` | `null` |
+| Transcription step failed | `true` | — | — | — |
+| Ollama unavailable, errored, or empty | `true` | — | — | — |
+
+A caller distinguishes "the user was silent" from "the user spoke but produced no words" by `speech_detected` (`false` vs `true`), both carrying `transcription: null`. Non-speech audio never reaches the language model: whisper's non-speech markers (`[BLANK_AUDIO]`, `[MUSIC]`, `(silence)`, and similar) are treated as no usable words rather than sent on as a query.
+
+**No speech** (`speech_detected: false`):
+
+```json
+{
+  "speech_detected": false,
+  "stopped_by": "no_speech_timeout",
+  "transcription": null,
+  "response": null,
+  "message": "No speech was detected. Ask the user to repeat, or check that the correct microphone is selected."
+}
+```
+
+**Speech, but no transcribable words** (`speech_detected: true`, `transcription: null`):
+
+```json
+{
+  "speech_detected": true,
+  "stopped_by": "silence",
+  "transcription": null,
+  "response": null,
+  "message": "Speech was detected but could not be transcribed. It may have been too quiet, too brief, or unclear. Ask the user to repeat, a little louder and closer to the microphone."
+}
+```
+
+Transcription and dependency failures return `isError: true` with the real cause (a missing model, a whisper load failure, Ollama not running, a timeout, or an empty model response), so a caller debugging can tell whether the failure was in capture, transcription, or the language model.
+
 ## How It Works
 
 mcp-listen uses [decibri](https://decibri.com) for cross-platform microphone capture. No ffmpeg, no SoX, no system audio tools required. Pre-built native binaries with zero setup.
 
 Audio is captured as 16-bit PCM at 16kHz mono, the standard format for speech-to-text engines.
 
+Silence-stopping uses the Silero voice activity detection model that ships inside decibri, running on-device through the bundled ONNX Runtime. Nothing extra is downloaded and no audio leaves the machine. The stop decision is measured in captured audio, not wall-clock time, and the VAD only decides when to stop: it never gates or alters the recorded samples.
+
 The `voice_query` tool runs the full pipeline locally: capture audio, transcribe with whisper.cpp, and send to a local Ollama LLM. Fully offline, nothing leaves your machine.
 
 ## Whisper Model Setup
@@ -194,7 +254,7 @@ The model is ~150MB and downloads once. You can also set the `WHISPER_MODEL_PATH
 
 ## Known Limitations
 
-1. **Fixed recording duration.** You specify how long to record. There is no "stop when I stop talking" mode yet.
+1. **A loud transient can register as speech.** Silence-stopping decides "speech has started" from the VAD score, so a door slam or a cough can start the countdown and end the recording after `silence_ms` of quiet, yielding a short capture of mostly silence. The outcome is visible, not silent: the result reports the actual duration, and `voice_query` reports an empty transcription rather than inventing one. A minimum-speech-duration guard is a candidate refinement.
 2. **`voice_query` requires Ollama running.** If Ollama isn't running, the tool returns a clear error message.
 3. **Whisper model must be downloaded before first use.** `voice_query` does not download the model itself; the first call requires a pre-downloaded model (~150MB). See [Whisper Model Setup](#whisper-model-setup).
 4. **No streaming.** MCP's request/response pattern means the entire recording is captured, then transcribed, then sent to the LLM. No real-time partial results.

SECURITY.md

@@ -81,8 +81,8 @@ This security policy applies to the following versions:
 
 | Version | Supported |
 | --- | --- |
-| 0.4.x | Yes |
-| < 0.4 | No |
+| 0.5.x | Yes |
+| < 0.5 | No |
 
 Security fixes are applied to the latest release only. mcp-listen is pre-1.0 and older versions are not backported. Upgrade to the latest release.