Add M2D dialogue background validation

Author: Michal Warda

README.md

@@ -106,6 +106,11 @@ uv run sam-pipeline-youtube-random \
   --total 1000 --seed 20260714
 ```
 
+Search metadata is only a candidate generator. Use the M2D validation command
+described in `docs/pipeline/OPERATIONS.md` before treating a clip as spoken
+dialogue over an active background. The gate explicitly rejects singing,
+rapping, choir, and other vocal music.
+
 Generation, prompt, batching, and TF32 policy are configured with the
 `SAM_AUDIO_*` environment variables demonstrated in
 [`deploy/start-sam-audio-api.sh`](deploy/start-sam-audio-api.sh). A supplied

docs/pipeline/OPERATIONS.md

@@ -205,6 +205,49 @@ uv run sam-pipeline-youtube-random \
   --total 1000 --verify-only
 ```
 
+### M2D dialogue/background validation
+
+The technical YouTube gate does not prove that spoken dialogue and background
+activity occur in the selected ten seconds. Validate candidates with the
+official M2D AudioSet-fine-tuned tagger before using them. The validator scores
+overlapping two-second windows and requires speech, a non-human background
+class, and at least three windows in which both are active. Singing, choir,
+chant, rapping, humming, opera, a capella, vocal music, and song evidence may
+appear in at most one window, preventing sung vocals from satisfying the speech
+requirement. It separately records music-led, effects/ambience-led, and mixed
+instrumental backgrounds.
+
+The model repository, checkpoint, AudioSet label CSV, and ontology are runtime
+inputs rather than files baked into this repository or its Docker image:
+
+```bash
+uv run sam-pipeline-m2d-validate score \
+  --input-dir /data/youtube-random-1000/audio \
+  --output /data/youtube-random-1000/m2d-validation.jsonl \
+  --m2d-repo /models/m2d \
+  --checkpoint /models/m2d/weights_ep69it3124-0.47929.pth \
+  --class-labels /models/audioset/class_labels_indices.csv \
+  --ontology /models/audioset/ontology.json \
+  --m2d-commit 3d0c4de9447c404a8d3f9f37e04f53bc902e09b3
+```
+
+Create a separate listening-test folder containing only accepted audio. The
+command hard-links files when possible and never removes the source dataset:
+
+```bash
+uv run sam-pipeline-m2d-validate materialize \
+  --input-dir ~/Downloads/youtube-random-1000-20260714/audio \
+  --results ~/Downloads/m2d-validation.jsonl \
+  --source-manifest ~/Downloads/youtube-random-1000-20260714/manifest.json \
+  --output-dir ~/Downloads/youtube-dialogue-background-m2d-ok-20260714
+```
+
+The output includes `manifest.json`, `audit.json`, the complete M2D JSONL, and
+the accepted WAVs under `audio/`. `balanced-audio/` is a deterministic listening
+subset with equal music-led and non-music-led counts. Every record keeps the
+per-window scores, ranks, top labels, temporal coverage, rejection reasons,
+exact M2D checkpoint hash, and validator policy version.
+
 ## AudioSet validation batches
 
 Acquire a reproducible random sample from the official AudioSet segment CSVs.

docs/pipeline/PROGRESS.md

@@ -122,6 +122,13 @@ was an example batch size, not a workflow limit.
   true-stereo, clipping, duration, uniqueness, existence, and SHA-256 gate
   passes. The audio payload is 1,920,078,000 bytes; no AudioSet record metadata
   or AudioSet candidate source was used.
+- M2D fragment validation scored all 1,000 local general-YouTube clips with the
+  official AudioSet-fine-tuned checkpoint. Policy
+  `spoken_dialogue_instrumental_background_m2d_v2` accepts 429 clips with
+  temporally overlapping spoken dialogue and active background, while rejecting
+  343 for singing, rapping, choir, or other vocal music. The listening output
+  preserves all 429 accepted clips and includes a 272-clip balanced subset with
+  136 music-led and 136 effects/ambience-or-mixed clips.
 - Live mono-filter job `5ec26b1bb28b420a9245e21d475d26be` completed with
   `non_stereo_input`, one input channel, zero chunks, zero stems, and zero model
   tasks. A live `targets=voice` API canary produced only voice and residual

pipeline/pyproject.toml

@@ -22,6 +22,7 @@ sam-pipeline-stereo-backfill = "sam_audio_pipeline.stereo:main"
 sam-pipeline-reconstruction-backfill = "sam_audio_pipeline.reconstruction:main"
 sam-pipeline-audio-profile-backfill = "sam_audio_pipeline.audio_profile:main"
 sam-pipeline-youtube-random = "sam_audio_pipeline.youtube_random:main"
+sam-pipeline-m2d-validate = "sam_audio_pipeline.m2d_validator:main"
 sam-audio-flamingo-api = "sam_audio_pipeline.flamingo_api:main"
 
 [dependency-groups]