Skip to content

Fix artifact schema drift and the Remotion render path - #459

Open
mozinio wants to merge 2 commits into
calesthio:mainfrom
mozinio:claude/brave-roentgen-4f5205
Open

Fix artifact schema drift and the Remotion render path#459
mozinio wants to merge 2 commits into
calesthio:mainfrom
mozinio:claude/brave-roentgen-4f5205

Conversation

@mozinio

@mozinio mozinio commented Jul 31, 2026

Copy link
Copy Markdown

Summary

Two related fixes. The artifact schemas had drifted from what the pipeline actually writes and reads, and the Remotion render path had three bugs that meant some of it had never worked at all.

The drift was invisible for a structural reason: every schema in schemas/artifacts/ sets additionalProperties: false, but video_compose never validates edit_decisions — only lib/checkpoint.py does, at checkpoint time. So a field the renderer happily consumed could be rejected long after the render already succeeded, and no test caught it because the tool tests call video_compose directly without validating the artifact.

Schema drift (33f48db)

edit_decisions was worst affected. Beyond cuts[].type and the per-scene payload fields, the schema also rejected:

  • cuts[].shot_language / shot_intent / narrative_role / information_role / hero_moment — read by _pre_compose_validation to feed slideshow_risk, so the pre-compose quality gate was scoring on fields the schema forbade
  • captions (word-level, ExplainerProps.captions), distinct from subtitles, which is the separate FFmpeg file-based burn config
  • theme / playbook / themeConfig, read by resolveTheme() in Root.tsx
  • overlays[] entirely — the schema wanted asset_id + start_seconds + a geometric position, while Explainer and TalkingHead read type + in_seconds/out_seconds + component props + a named slot. Both shapes validate now.
  • end_tag, written by documentary-montage's edit-director and read by its compose-director — a pure cross-stage contract

Other schemas:

  • asset_manifest required scene_id on every asset, so background music could never validate. Now optional, documented as required for scene-bound assets.
  • final_review enforced frames_sampled >= 4 while the producer explicitly handles fewer and records it as an issue — a partially-failed frame extraction produced a review the checkpoint writer then refused.
  • render_report gained render_runtime, composition_mode, render_summary, audio_channels.
  • scene_plan gained the overlay-scene shape talking-head uses across stages.

Stale director-skill examples were fixed to match their schemas: the explainer render_report used file_size_mb and a per-output render_time_seconds; the asset manifest put generation_summary and music_status at the root instead of under metadata; documentary-montage's brief was written flat when every downstream director reads brief.metadata.*.

proposal_packet was clean.

Remotion render path (a7af02d)

Asset resolution. _remotion_render rewrote on-disk cut sources into file:// URLs, but every component resolves media through a local resolveAsset() ending in staticFile(), which only understands paths relative to public/ — it prefixes whatever it is given, producing http://localhost:3000/public/Users/... and a 404. AnimeScene's copy doesn't even special-case absolute paths, so it failed that way for every local file. Replaced with _stage_remotion_assets, which copies referenced assets into a per-render dir under public/, rewrites public-relative, and cleans up in finally. Traversal is scoped per container rather than matching key names anywhere in the tree, because subtitles.source is an .srt for the FFmpeg burn and must not be touched.

CinematicRenderer. renderer_family of cinematic-trailer or documentary-montage routes to a composition whose props are a different shape entirely — scenes[] of {kind, startSeconds, durationSeconds} plus soundtrack/music/captions.words — but edit_decisions was passed straight through, so calculateMetadata threw on props.scenes.length and this path had never worked. _to_cinematic_props adapts cuts to it. The adapter lives in the tool rather than the artifact because cuts is the cross-stage contract that backlot/state.py, slideshow_risk and _pre_compose_validation all read; HyperFrames already adapts the same artifact for its runtime. Cut types the composition can't express fail loudly with the ways out, since dropping them would silently change the deliverable.

Audio. The asset-id spelling documented for FFmpeg and HyperFrames had no resolver on the Remotion path, so documentary-montage — which is required to run on Remotion — rendered silent. Multi-segment narration is refused with a pointer to audio_mixer rather than shipping the first segment of five.

Also corrects _REMOTION_SCENE_TYPES, which listed "progress" and "chart" that no component ever matched, while missing all nine chart and synthetic-UI types added since.

Verification

  • Real render, not just a typecheck: a documentary-montage edit_decisions produced a 9.0s 1920x1080 h264 file with an aac stream present — the tell that asset-id audio resolution works. Sampled frames confirm the correct clip per cut, the tone treatment, word-level captions, and the title card.
  • npx tsc --noEmit before vs. after: byte-identical error lists. The 15 pre-existing errors live in Root.tsx/Explainer.tsx/ProviderChip.tsx/TitledVideo.tsx; the two files touched here are clean.
  • 831 passed, 9 skipped. The first commit was checked out on its own and is green (802 passed) — no broken intermediate state.
  • The new tests were verified to go red against the old schema (17 failures), so they genuinely catch this.

Regression guard

tests/contracts/test_artifact_schema_drift.py pins the scene-type registry across four sources — Explainer.tsx's dispatch cases, SCENE_TYPES.md, the schema enum, and _REMOTION_SCENE_TYPES — so a type can never again be routable but unrepresentable. It also asserts every complete artifact example in the director skills validates, which is how the render_report and documentary-montage examples silently rotted. SCENE_TYPES.md's "adding a new scene type" checklist now includes the schema and the Python set.

Reviewer notes

  • Schema changes are purely additive widening plus doc corrections — nothing that previously validated stops validating. cuts[].type is a closed enum, not a free string, so typos like text-card are still rejected.
  • One judgement call worth a second opinion: unsupported cut types on the cinematic path fail the render rather than being skipped. That's the governance-correct reading (silently dropping a cut changes the deliverable), but it is stricter than before.

🤖 Generated with Claude Code

mohit pursani and others added 2 commits July 31, 2026 08:22
Every schema in schemas/artifacts/ sets additionalProperties: false, but
video_compose never validates edit_decisions — only lib/checkpoint.py does,
at checkpoint time. So a field the renderer happily consumes could be
rejected long after the render already succeeded, and no test caught it
because the tool tests call video_compose directly.

edit_decisions was the worst affected. Beyond cuts[].type and the per-scene
payload fields, the schema also rejected:

  - cuts[].shot_language / shot_intent / narrative_role / information_role /
    hero_moment, which _pre_compose_validation reads to feed slideshow_risk —
    the pre-compose gate was scoring on fields the schema forbade
  - captions (word-level, ExplainerProps.captions) as distinct from subtitles,
    which is the separate FFmpeg file-based burn config
  - theme / playbook / themeConfig, read by resolveTheme() in Root.tsx
  - overlays[] entirely: the schema wanted asset_id + start_seconds and a
    geometric position, while Explainer and TalkingHead read type +
    in_seconds/out_seconds + component props and a named slot. Both shapes
    now validate.
  - end_tag, written by documentary-montage's edit-director and read by its
    compose-director — a pure cross-stage contract

Other schemas:

  - asset_manifest required scene_id on every asset, so background music
    could never validate. Now optional, documented as required for
    scene-bound assets.
  - final_review enforced frames_sampled >= 4 while the producer explicitly
    handles fewer and records it as an issue, so a partially-failed frame
    extraction produced a review the checkpoint writer then refused.
  - render_report gained render_runtime, composition_mode, render_summary
    and audio_channels.
  - scene_plan gained the overlay-scene shape talking-head uses across
    stages.

Stale director-skill examples fixed to match their schemas: the explainer
render_report used file_size_mb and a per-output render_time_seconds, the
asset manifest put generation_summary and music_status at the root instead
of under metadata, and documentary-montage's brief was written flat when
every downstream director reads brief.metadata.*.

proposal_packet was clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…enderer

Three bugs kept renders from working, plus a regression guard for the
schema drift fixed in the previous commit.

Asset resolution. _remotion_render rewrote any cut source that resolved on
disk into a file:// URL, but every component resolves media through a local
resolveAsset() that ends in staticFile(), which only understands paths
relative to public/ — it prefixes whatever it is given, producing requests
like http://localhost:3000/public/Users/... that 404. AnimeScene's copy of
resolveAsset() does not even special-case absolute paths, so it failed that
way for every local file. Replaced with _stage_remotion_assets, which copies
each referenced asset into a per-render dir under public/ and rewrites the
reference public-relative, then cleans up in finally. Traversal is scoped
per container rather than matching key names anywhere in the tree, because
subtitles.source is an .srt for the FFmpeg burn and must not be touched.

CinematicRenderer. renderer_family cinematic-trailer and documentary-montage
route to a composition whose props are a different shape entirely — scenes[]
of {kind, startSeconds, durationSeconds} plus soundtrack/music/captions.words
— but edit_decisions was passed straight through, so calculateMetadata threw
on props.scenes.length and the path had never worked. _to_cinematic_props
adapts cuts to it: media cuts become video scenes with source_in_seconds as
trimBeforeSeconds, text_card/hero_title become title cards, and fade-like
transitions become frame ramps while "cut" pins them to zero. The adapter
lives in the tool rather than the artifact because cuts is the cross-stage
contract that backlot, slideshow_risk and _pre_compose_validation all read;
HyperFrames already adapts the same artifact for its runtime. Cut types the
composition cannot express fail loudly with the ways out, since dropping
them would silently change the deliverable.

Audio. The asset-id spelling documented for FFmpeg and HyperFrames had no
resolver on the Remotion path, so documentary-montage — which is required to
run on Remotion — rendered silent. _resolve_audio_sources fills in src from
asset_id. Multi-segment narration is refused with a pointer to audio_mixer
rather than shipping the first segment of five.

Also corrects _REMOTION_SCENE_TYPES, which listed "progress" and "chart"
that no component ever matched while missing all nine chart and
synthetic-UI types added since.

Verified with a real render: a documentary-montage edit_decisions produced a
9.0s 1920x1080 h264 file with an aac stream (proving asset-id resolution),
and sampled frames show the correct clip per cut, the tone treatment, the
word-level captions and the title card. tsc --noEmit reports no new errors.

The new contract test pins the scene-type registry across four sources —
Explainer's dispatch cases, SCENE_TYPES.md, the schema enum and the Python
set — so a type can never again be routable but unrepresentable, and checks
that every complete artifact example in the director skills validates.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant