Add general YouTube dataset sampler

Author: Michal Warda

README.md

@@ -93,6 +93,19 @@ data** dashboard. Each original also reports mono/stereo layout, codec/container
 sample rate, bit depth, bitrate, and a lossless/lossy quality tier. The direct
 model ZIP API remains unchanged.
 
+A separate resumable sampler can build private, mix-biased datasets directly
+from general YouTube search (not AudioSet). It downloads only a short source
+section, produces an exact ten-second PCM16/48 kHz stereo WAV, and rejects low
+source bitrate/sample rate, duplicated mono, quiet or silent audio, clipping,
+and duration errors:
+
+```bash
+cd pipeline
+uv run sam-pipeline-youtube-random \
+  --output ~/Downloads/youtube-random-1000-20260714 \
+  --total 1000 --seed 20260714
+```
+
 Generation, prompt, batching, and TF32 policy are configured with the
 `SAM_AUDIO_*` environment variables demonstrated in
 [`deploy/start-sam-audio-api.sh`](deploy/start-sam-audio-api.sh). A supplied

docs/pipeline/OPERATIONS.md

@@ -166,6 +166,45 @@ Dashboard state is encoded in clean paths and is safe to refresh or share:
 Selecting a tab, job, dataset, or sound updates browser history. Back and
 forward navigation restore the corresponding view and selected record.
 
+## General YouTube random datasets
+
+This sampler searches YouTube directly and does not read AudioSet manifests or
+timestamps. Randomized, reproducible queries favor program audio containing
+voices, music, and environmental effects while excluding obvious music-only,
+sleep, ambience, and playlist results. Each video ID contributes at most one
+clip.
+
+```bash
+cd pipeline
+uv run sam-pipeline-youtube-random \
+  --output ~/Downloads/youtube-random-1000-20260714 \
+  --total 1000 \
+  --seed 20260714 \
+  --query-count 400 \
+  --results-per-query 15 \
+  --search-workers 8 \
+  --download-workers 8 \
+  --candidate-multiplier 2 \
+  --max-attempts 2500
+```
+
+Acquisition is resumable. `attempts.jsonl` records every accepted, rejected, or
+unavailable candidate; `metadata/candidates.json` and `metadata/search.json`
+preserve search provenance; `manifest.json` contains only accepted records and
+hashes. `audit.json` is written only after reopening and verifying the complete
+dataset. The gate requires source audio at 44.1 kHz or better and at least 120
+kbps, then verifies exact ten-second PCM16/48 kHz stereo output, real left/right
+difference, loudness, silent-frame/run limits, clipping, uniqueness, and SHA-256.
+
+Re-run the same command to continue an interrupted build. Audit an existing set
+without downloading:
+
+```bash
+uv run sam-pipeline-youtube-random \
+  --output ~/Downloads/youtube-random-1000-20260714 \
+  --total 1000 --verify-only
+```
+
 ## AudioSet validation batches
 
 Acquire a reproducible random sample from the official AudioSet segment CSVs.

docs/pipeline/PROGRESS.md

@@ -38,6 +38,9 @@ was an example batch size, not a workflow limit.
   facts, with an idempotent historical backfill.
 - [x] Give pipeline, review, dataset, job, and source dashboard states durable
   URLs with refresh and browser-history restoration.
+- [x] Add a resumable general-YouTube (non-AudioSet) sampler with deterministic
+  discovery, exact ten-second cuts, stereo/source-quality/silence gates,
+  per-attempt provenance, hashes, and a full-dataset verifier.
 - [x] Require stereo input, gate SAM targets with source-scene presence, support
   single-stage inference, and use identity pass-through for pure SFX sources.
 - [x] Add persistent datasets, successive upload jobs, and reconciliation of

pipeline/pyproject.toml

@@ -21,6 +21,7 @@ sam-pipeline-submit = "sam_audio_pipeline.submit:main"
 sam-pipeline-stereo-backfill = "sam_audio_pipeline.stereo:main"
 sam-pipeline-reconstruction-backfill = "sam_audio_pipeline.reconstruction:main"
 sam-pipeline-audio-profile-backfill = "sam_audio_pipeline.audio_profile:main"
+sam-pipeline-youtube-random = "sam_audio_pipeline.youtube_random:main"
 sam-audio-flamingo-api = "sam_audio_pipeline.flamingo_api:main"
 
 [dependency-groups]