Skip to content

CLI.md

Latest commit

 

History

History
197 lines (150 loc) · 7.81 KB

CLI.md

File metadata and controls

197 lines (150 loc) · 7.81 KB
Raw

Command-line interface

tools/ runs the same pipeline as the ComfyUI nodes, without ComfyUI. Both call the same functions in memoacts_core.pipeline with the same arguments, so the output is identical.

Use it for scripting, batch runs, and for comparing node output against a known-good result. The September workshop teaches the node graph instead — see docs/WORKSHOP_HANDOUT.md.

Requirements: Python 3.10 or later, stable-ts, num2words, and ffmpeg built with libass. ComfyUI is not required. All commands run from the repository root.

0. Every session: activate the virtual environment first

cd ~/Documents/MemoActs/MemoActStudio
source .venv/bin/activate

The prompt changes to (.venv) …. This is per terminal window — a new tab starts without it, and then python is macOS's own interpreter, which has none of the dependencies. That is what a ModuleNotFoundError: stable_whisper means, every time.

Check it took:

python -c "import stable_whisper, num2words, numpy, PIL; print('ok')"

deactivate leaves the environment when you are done.

1. One-time setup on a new machine

brew tap homebrew-ffmpeg/ffmpeg          # see below — not the core formula
brew install homebrew-ffmpeg/ffmpeg/ffmpeg
python3 -m venv .venv
source .venv/bin/activate
pip install stable-ts num2words numpy pillow

Not brew install ffmpeg. Homebrew's core formula no longer lists libass among its dependencies, so the ffmpeg it installs has no subtitles filter at all, and the render dies with No such filter: 'subtitles' — after every frame has been generated, which is the expensive place to find out. The homebrew-ffmpeg tap builds it in by default. If the core formula is already installed, brew unlink ffmpeg first so the tap's binary is the one on PATH.

Verify before rendering anything, not after:

ffmpeg -version | grep -o libass         # must print libass
ffmpeg -filters | grep -w subtitles      # must print one line

stable-ts pulls in torch, and the Whisper model downloads on first alignment (a few hundred MB, cached in ~/.cache/whisper). It is the only heavy dependency.

2. Put the media in place

Media is never versioned, so it arrives separately:

projects/legends_of_surrender/
    script.md            the text — ground truth
    shots.csv            the edit decision
    REBUILD.md           how every generated file is remade
    sources/             everything the edit points at (never versioned)
        SOURCES.md       provenance and rights
        narration.wav    the read
        images/          stills
        videos/          MBK_KAPFILM_FINAL.mp4
        composites/      page moves and stacked frames
        maps/            drawn plates
    generated/           shots.json + report.txt — delete and remake at will
    out/                 the finished reel
    archive/             superseded, kept for its reasoning

Three folders and three files: the project root holds the decisions, sources/ holds everything those decisions name, and generated/ and out/ hold what a command can always make again.

Exactly one narration.* file belongs in sources/. The generator takes whichever narration.* it finds there, falling back to the project root for older fixtures — two files make the choice arbitrary. Keep the others in archive/.

3. Rebuild the generated clips

Maps and stacked-frame composites are outputs, not sources: they are rebuilt rather than copied, so they cannot silently disagree with the images they claim to come from. Both must exist before a render.

python tools/render_map.py --out projects/legends_of_surrender/sources/maps \
    --name map_baltics --frames 360 \
    --highlight Latvia Estonia Lithuania

python tools/render_map.py --out projects/legends_of_surrender/sources/maps \
    --name map_poland_ukraine --frames 360 \
    --highlight Poland Ukraine --already Latvia Estonia Lithuania

python tools/render_bands.py --project projects/legends_of_surrender \
    --name S14_three-band --frames 360 \
    --band GIoS_Wehrmacht_Signed_Ru.jpg:0.277:zoom_in:0.08 \
    --band Wehrmacht_in_Karlshorst.jpg:0.482:pan_lr:0.10 \
    --band GIoS_Wehrmacht_Signed_En.jpg:0.530:zoom_out:0.08

360 frames is 12 s at 30 fps — longer than any of those shots needs. A clip shorter than its shot is an error, not a freeze.

4. Align the narration

python tools/generate_shots.py --project projects/legends_of_surrender --lang en

Reads script.md and sources/narration.wav; writes generated/shots.json, generated/crops/*.csv and generated/report.txt. Nothing is transcribed — the script is ground truth and alignment computes timings only.

5. Read the report before rendering

open projects/legends_of_surrender/generated/report.txt

Three things to look at:

meaning
confidence mean word probability per shot. Low means the read and the script disagree there.
[DRIFT ±Ns] how far the aligned start sits from the cue written in script.md. Cues are advisory; alignment wins. A large drift means the cues are stale, not that the timing is wrong.
[CLAMPED], x… enlargement properties of the images, not of the read. A source too small for the frame.

shots.json is a plain file, meant to be read and edited between the two steps.

6. Render

python tools/render_reel.py --project projects/legends_of_surrender

Writes out/reel.mp4 at 1080×1920, 30 fps, with the .ass and .srt tracks beside it. Useful flags: --no-subs, --sub-size 56, --plate 0.80 (subtitle plate opacity), --on-upscale warn|error|allow, --crf 19.

--shot 7 renders that shot alone — seconds instead of minutes, which is the way to judge a framing or a move without re-rendering the reel. It is a preview and says so: no narration and no captions, because both are timed from the head of the reel and would be wrong against a fragment of it.

Each shot's look comes from its own effects column in shots.csv; --effects archive_soft sets the look for every shot that names none. Effects cost three to four times the render time, so a whole reel under a heavy preset is minutes, not seconds — check one shot first.

7. Re-recording the narration

Replace sources/narration.wav and re-run steps 4 and 6. Nothing else moves: alignment recomputes every shot boundary and every word timing, captions are cut at the new word timings, and shots.csv is keyed to script cues rather than to seconds. No timing is stored anywhere that survives the new read.

When it goes wrong

symptom cause
ModuleNotFoundError: stable_whisper the environment is not active — step 0
command not found: python same; macOS ships python3 only
ffmpeg: command not found step 1
No such filter: 'subtitles' ffmpeg built without libass — step 1, and note it is not the core Homebrew formula
No option name near … in a filter ffmpeg 8 rejects quoted filter values; memoacts_core.render escapes them instead — update the repository
every shot reports cue … matches no block in script.md shots.csv keys no longer match script.md; re-key one of the two
confidence is 0.00 everywhere alignment fell back to proportional timing; the log above says why
a shot renders as its default its media is missing — the generator warns by name and carries on

Known gap

shots.csv references three stacked-frame stills — S01-02_two-band.png, S07_two-band_bw.png, S18_three-cities_bw.png — that no script in this repository builds. They were made in ComfyUI with Olm-DragCrop on the Windows install (docs/THREEBAND_TOOL.md). Until they are either copied across or render_bands.py learns to emit stills, those three shots fall back to their defaults and the render warns.