Skip to content

Commit 7d94c80

Browse files
videoeditor-record: web-based narration recorder (videoeditor record) (#9)
* feat: videoeditor-record — record narration takes in a local web recorder New crate videoeditor-record and a `record` subcommand: `videoeditor record <episode>` serves a one-page teleprompter + mic recorder on 127.0.0.1 and writes kept takes straight into the episode, ready for the normal render/assemble pipeline. - Capture in the browser (getUserMedia/MediaRecorder): device picker, permission UX, and live level metering are already solved there — cross-platform native capture is not. localhost is a secure context, so the mic works without TLS. Raw voice: echoCancellation / noiseSuppression / autoGainControl all disabled. - UI: clip sidebar with per-take fit status, big teleprompter text, 3-2-1 countdown, elapsed-vs-window timer that flags overruns live, level meter (WebAudio), review player, keyboard-driven flow (space/enter/r/arrows). - Server (tiny_http, sync — no async runtime added): uploads (webm/opus or mp4/aac) transcode via ffmpeg to the exact mp3 format the TTS path produces; every kept take lands in audio/takes/<id>/ and the replaced clip is archived, so no take is ever lost; audio/clips.json is rebuilt from disk (self-healing) and each save returns the episode fit-check. - Clip ids off the URL are allowlist-sanitized (no path traversal). - guide.md pipeline/director-loop + CLAUDE.md layout updated. Live-tested end to end against a scratch episode: webm upload → transcode → archive (take_001/take_002/replaced_003) → manifest → fit-check response; traversal attempts rejected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(record): take coaching — level checks + ElevenLabs Scribe review before you keep a take POST /api/review/<id> analyzes a pending take without keeping it, and the UI shows a COACH panel while you listen back: - always (no API needed): duration vs scene window, mean/max dBFS via ffmpeg volumedetect, clipping and too-quiet flags - with ELEVENLABS_API_KEY: Scribe transcription → script-vs-spoken word diff (LCS; dropped words + ad-libs), words/sec pacing, dead-air gaps from word timestamps, tagged background audio events - coaching lines synthesize it ("dropped from the script: boomer", "dead air at 3.0s", "clipping — back off the mic", or "clean take — ship it"); degrades gracefully to local-only metrics without a key Tests: LCS diff (missing/added words), clean-take praise, multi-fault coaching (overrun + clipping + drops + racing + dead air + cough). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(record): stereo capture + archive every take - capture and transcode stereo (interface sends a stereo mix; assembly normalizes all narration to stereo anyway), 192k mp3 archive quality - L/R channel meters in the UI instead of a single downmixed bar - every take is archived to audio/takes/<id>/take_NNN.mp3 the moment it is reviewed — retakes lose nothing; keep promotes the archived file via /api/keep/<id>/<n> without re-uploading Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(record): takes browser + per-clip approvals - GET /api/takes/<id> lists every archived take (take_* and replaced_*) with duration and approval state; GET /audio/takes/<id>/<file> serves them for audition - POST /api/approve/<id>/<file> promotes an archived take to the clip and records the pick in audio/takes/approvals.json (in-episode, so approvals survive restarts and travel with the repo) - keep after recording is now just an approval of the freshly archived take; A/B switching approved takes doesn't pile up duplicate archives - UI: takes panel under the prompter with ▶ audition and approve buttons, approved take highlighted Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(record): persist each take's coach report beside the audio - review saves take_NNN.json next to take_NNN.mp3 — the archive holds the analysis, not just the audio - /api/takes/<id> returns each take's saved review; the takes panel shows comparison stats per row (script %, pace, peak, clipped, long, pauses) with full coaching notes on hover - takes panel refreshes as soon as a new take is analyzed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(record): studio layout + modular front-end - three-column layout: clips | takes rail | stage — takes and coach move out of the teleprompter area for good - state-aware stage via body class: recording dims the rails so only the script competes for attention; review shrinks (never hides) the script and shows one review card: coach stats, notes, transcript, player, keep/retake - takes rail newest-first (list_takes sorts descending) - uber index.html split into web/: index.html + style.css + 6 plain ES modules (state/mic/record/review/takes/main) talking over an event bus — no framework, no build step, still embedded via include_str! Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(record): inline take review via progressive disclosure Tap a take row to unfold its saved coach report — coaching notes + transcript — right in the rail (chevron rotates, height animates via the grid 0fr→1fr trick, one row open at a time). Replaces the hidden hover tooltip. Play/approve stop propagation so they don't toggle. Expansion is keyed clip/file so re-renders keep it open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(record)!: Leptos front end + nix-first toolchain + playwright e2e The recorder UI is now Rust end to end: crates/videoeditor-record-ui is a Leptos 0.8 (CSR) app compiled to wasm with trunk. videoeditor-record embeds the COMMITTED dist (ui/, regenerate with `just ui`) so the CLI stays one self-contained binary and both crates publish to crates.io with no wasm toolchain required by consumers. - same REST API, same stylesheet contract, same studio layout: clips | takes rail (newest first, inline disclosure of the saved coach report) | teleprompter stage that shrinks-but-never-hides the script during review; body-class state machine unchanged - audio interop (getUserMedia, stereo ChannelSplitter analysers, MediaRecorder) via web-sys; wasm-bindgen pinned =0.2.121 to match nixpkgs' wasm-bindgen-cli (bump together) - nix devshell now carries the whole toolchain: rust w/ wasm32 target (rust-overlay), trunk, wasm-bindgen-cli, node, playwright + pinned browsers (PLAYWRIGHT_BROWSERS_PATH); nix build needs no wasm because the dist is committed - e2e/: 9-test playwright journey against the real binary + real Chromium with a fake mic — boot, record→coach→keep, newest-first, disclosure, approve-from-rail, audition, retake, clip nav, and the script-never-hidden guarantee; `just e2e`, wired into CI (ubuntu) - vanilla JS front end deleted Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(record): stable UI pipeline — private UI crate, auto-rebuilt dist, drift guard - videoeditor-record-ui is publish = false (+ release-plz release = false): it doesn't stand alone; its trunk build ships inside videoeditor-record's committed ui/ dist, which publishes to crates.io as part of that crate - no manual steps: `just e2e` depends on `just ui` (dist always rebuilt from source before the suite), and `just check` — which CI runs on every push, no wasm toolchain required — verifies ui/.source-hash against the UI source, so a stale committed dist cannot merge - dist regenerated with the nix toolchain so bytes stay stable across machines; docs updated to match Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent d225924 commit 7d94c80

34 files changed

Lines changed: 5424 additions & 15 deletions

.github/workflows/ci.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,19 @@ jobs:
3232
- name: build
3333
run: just build
3434

35+
# recorder UI end-to-end: real binary, real Chromium, fake mic — the whole
36+
# record → coach → keep → approve journey (see e2e/)
37+
record-e2e:
38+
runs-on: ubuntu-latest
39+
steps:
40+
- uses: actions/checkout@v4
41+
with:
42+
lfs: true
43+
- uses: DeterminateSystems/nix-installer-action@v16
44+
- uses: DeterminateSystems/magic-nix-cache-action@v9
45+
- name: playwright e2e
46+
run: nix develop --command just e2e
47+
3548
# the preferred install path must stay green: sandboxed build from flake.lock
3649
nix:
3750
strategy:

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,9 @@ analysis/
1919

2020
# videoeditor preview output
2121
template-previews/
22+
23+
# trunk build output (the committed copy lives in crates/videoeditor-record/ui)
24+
crates/videoeditor-record-ui/dist/
25+
# playwright artifacts
26+
e2e/test-results/
27+
e2e/playwright-report/

CLAUDE.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,21 @@ vertical video out via headless Chrome + ffmpeg + ElevenLabs).
1616
- `crates/videoeditor-genai` — typed image-generation clients: xAI Grok
1717
Imagine (`XAI_API_KEY`, reference images) + Google Imagen (`AI_STUDIO`);
1818
Veo/Grok video is the planned next tenant.
19+
- `crates/videoeditor-record``record` subcommand: local web recorder
20+
(tiny_http REST server); takes transcode via ffmpeg into `audio/clips/`.
21+
Serves the UI from `ui/` — a COMMITTED trunk build of the next crate.
22+
Never edit `ui/` by hand; `just e2e` regenerates it automatically and
23+
`just check` fails if it's stale vs the UI source (`ui/.source-hash`).
24+
This crate publishes to crates.io WITH the dist — installs need no wasm.
25+
- `crates/videoeditor-record-ui` — the recorder front end: Leptos (CSR) →
26+
wasm via trunk. `publish = false`: not standalone, its build output
27+
ships inside videoeditor-record. `nix develop` provides
28+
trunk/wasm-bindgen-cli (the `wasm-bindgen` crate pin must match the CLI
29+
version — bump together). Regenerate the dist only via the nix
30+
toolchain (`nix develop --command just ui`) so bytes stay stable.
31+
- `e2e/` — Playwright suite for the recorder (`just e2e`, rebuilds the UI
32+
first): drives the real binary + real Chromium with a fake mic through
33+
record → coach → keep → approve. UI changes are not done until it passes.
1934
- `examples/hello-bench` — smallest end-to-end episode; keep it rendering.
2035

2136
## Commands

0 commit comments

Comments
 (0)