Fix artifact schema drift and the Remotion render path - #459
Open
mozinio wants to merge 2 commits into
Open
Conversation
Every schema in schemas/artifacts/ sets additionalProperties: false, but
video_compose never validates edit_decisions — only lib/checkpoint.py does,
at checkpoint time. So a field the renderer happily consumes could be
rejected long after the render already succeeded, and no test caught it
because the tool tests call video_compose directly.
edit_decisions was the worst affected. Beyond cuts[].type and the per-scene
payload fields, the schema also rejected:
- cuts[].shot_language / shot_intent / narrative_role / information_role /
hero_moment, which _pre_compose_validation reads to feed slideshow_risk —
the pre-compose gate was scoring on fields the schema forbade
- captions (word-level, ExplainerProps.captions) as distinct from subtitles,
which is the separate FFmpeg file-based burn config
- theme / playbook / themeConfig, read by resolveTheme() in Root.tsx
- overlays[] entirely: the schema wanted asset_id + start_seconds and a
geometric position, while Explainer and TalkingHead read type +
in_seconds/out_seconds + component props and a named slot. Both shapes
now validate.
- end_tag, written by documentary-montage's edit-director and read by its
compose-director — a pure cross-stage contract
Other schemas:
- asset_manifest required scene_id on every asset, so background music
could never validate. Now optional, documented as required for
scene-bound assets.
- final_review enforced frames_sampled >= 4 while the producer explicitly
handles fewer and records it as an issue, so a partially-failed frame
extraction produced a review the checkpoint writer then refused.
- render_report gained render_runtime, composition_mode, render_summary
and audio_channels.
- scene_plan gained the overlay-scene shape talking-head uses across
stages.
Stale director-skill examples fixed to match their schemas: the explainer
render_report used file_size_mb and a per-output render_time_seconds, the
asset manifest put generation_summary and music_status at the root instead
of under metadata, and documentary-montage's brief was written flat when
every downstream director reads brief.metadata.*.
proposal_packet was clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…enderer
Three bugs kept renders from working, plus a regression guard for the
schema drift fixed in the previous commit.
Asset resolution. _remotion_render rewrote any cut source that resolved on
disk into a file:// URL, but every component resolves media through a local
resolveAsset() that ends in staticFile(), which only understands paths
relative to public/ — it prefixes whatever it is given, producing requests
like http://localhost:3000/public/Users/... that 404. AnimeScene's copy of
resolveAsset() does not even special-case absolute paths, so it failed that
way for every local file. Replaced with _stage_remotion_assets, which copies
each referenced asset into a per-render dir under public/ and rewrites the
reference public-relative, then cleans up in finally. Traversal is scoped
per container rather than matching key names anywhere in the tree, because
subtitles.source is an .srt for the FFmpeg burn and must not be touched.
CinematicRenderer. renderer_family cinematic-trailer and documentary-montage
route to a composition whose props are a different shape entirely — scenes[]
of {kind, startSeconds, durationSeconds} plus soundtrack/music/captions.words
— but edit_decisions was passed straight through, so calculateMetadata threw
on props.scenes.length and the path had never worked. _to_cinematic_props
adapts cuts to it: media cuts become video scenes with source_in_seconds as
trimBeforeSeconds, text_card/hero_title become title cards, and fade-like
transitions become frame ramps while "cut" pins them to zero. The adapter
lives in the tool rather than the artifact because cuts is the cross-stage
contract that backlot, slideshow_risk and _pre_compose_validation all read;
HyperFrames already adapts the same artifact for its runtime. Cut types the
composition cannot express fail loudly with the ways out, since dropping
them would silently change the deliverable.
Audio. The asset-id spelling documented for FFmpeg and HyperFrames had no
resolver on the Remotion path, so documentary-montage — which is required to
run on Remotion — rendered silent. _resolve_audio_sources fills in src from
asset_id. Multi-segment narration is refused with a pointer to audio_mixer
rather than shipping the first segment of five.
Also corrects _REMOTION_SCENE_TYPES, which listed "progress" and "chart"
that no component ever matched while missing all nine chart and
synthetic-UI types added since.
Verified with a real render: a documentary-montage edit_decisions produced a
9.0s 1920x1080 h264 file with an aac stream (proving asset-id resolution),
and sampled frames show the correct clip per cut, the tone treatment, the
word-level captions and the title card. tsc --noEmit reports no new errors.
The new contract test pins the scene-type registry across four sources —
Explainer's dispatch cases, SCENE_TYPES.md, the schema enum and the Python
set — so a type can never again be routable but unrepresentable, and checks
that every complete artifact example in the director skills validates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related fixes. The artifact schemas had drifted from what the pipeline actually writes and reads, and the Remotion render path had three bugs that meant some of it had never worked at all.
The drift was invisible for a structural reason: every schema in
schemas/artifacts/setsadditionalProperties: false, butvideo_composenever validatesedit_decisions— onlylib/checkpoint.pydoes, at checkpoint time. So a field the renderer happily consumed could be rejected long after the render already succeeded, and no test caught it because the tool tests callvideo_composedirectly without validating the artifact.Schema drift (
33f48db)edit_decisionswas worst affected. Beyondcuts[].typeand the per-scene payload fields, the schema also rejected:cuts[].shot_language/shot_intent/narrative_role/information_role/hero_moment— read by_pre_compose_validationto feedslideshow_risk, so the pre-compose quality gate was scoring on fields the schema forbadecaptions(word-level,ExplainerProps.captions), distinct fromsubtitles, which is the separate FFmpeg file-based burn configtheme/playbook/themeConfig, read byresolveTheme()inRoot.tsxoverlays[]entirely — the schema wantedasset_id+start_seconds+ a geometric position, whileExplainerandTalkingHeadreadtype+in_seconds/out_seconds+ component props + a named slot. Both shapes validate now.end_tag, written by documentary-montage's edit-director and read by its compose-director — a pure cross-stage contractOther schemas:
asset_manifestrequiredscene_idon every asset, so background music could never validate. Now optional, documented as required for scene-bound assets.final_reviewenforcedframes_sampled >= 4while the producer explicitly handles fewer and records it as an issue — a partially-failed frame extraction produced a review the checkpoint writer then refused.render_reportgainedrender_runtime,composition_mode,render_summary,audio_channels.scene_plangained the overlay-scene shape talking-head uses across stages.Stale director-skill examples were fixed to match their schemas: the explainer
render_reportusedfile_size_mband a per-outputrender_time_seconds; the asset manifest putgeneration_summaryandmusic_statusat the root instead of undermetadata; documentary-montage's brief was written flat when every downstream director readsbrief.metadata.*.proposal_packetwas clean.Remotion render path (
a7af02d)Asset resolution.
_remotion_renderrewrote on-disk cut sources intofile://URLs, but every component resolves media through a localresolveAsset()ending instaticFile(), which only understands paths relative topublic/— it prefixes whatever it is given, producinghttp://localhost:3000/public/Users/...and a 404.AnimeScene's copy doesn't even special-case absolute paths, so it failed that way for every local file. Replaced with_stage_remotion_assets, which copies referenced assets into a per-render dir underpublic/, rewrites public-relative, and cleans up infinally. Traversal is scoped per container rather than matching key names anywhere in the tree, becausesubtitles.sourceis an.srtfor the FFmpeg burn and must not be touched.CinematicRenderer.
renderer_familyofcinematic-trailerordocumentary-montageroutes to a composition whose props are a different shape entirely —scenes[]of{kind, startSeconds, durationSeconds}plussoundtrack/music/captions.words— butedit_decisionswas passed straight through, socalculateMetadatathrew onprops.scenes.lengthand this path had never worked._to_cinematic_propsadaptscutsto it. The adapter lives in the tool rather than the artifact becausecutsis the cross-stage contract thatbacklot/state.py,slideshow_riskand_pre_compose_validationall read; HyperFrames already adapts the same artifact for its runtime. Cut types the composition can't express fail loudly with the ways out, since dropping them would silently change the deliverable.Audio. The asset-id spelling documented for FFmpeg and HyperFrames had no resolver on the Remotion path, so documentary-montage — which is required to run on Remotion — rendered silent. Multi-segment narration is refused with a pointer to
audio_mixerrather than shipping the first segment of five.Also corrects
_REMOTION_SCENE_TYPES, which listed"progress"and"chart"that no component ever matched, while missing all nine chart and synthetic-UI types added since.Verification
edit_decisionsproduced a 9.0s 1920x1080 h264 file with an aac stream present — the tell that asset-id audio resolution works. Sampled frames confirm the correct clip per cut, the tone treatment, word-level captions, and the title card.npx tsc --noEmitbefore vs. after: byte-identical error lists. The 15 pre-existing errors live inRoot.tsx/Explainer.tsx/ProviderChip.tsx/TitledVideo.tsx; the two files touched here are clean.Regression guard
tests/contracts/test_artifact_schema_drift.pypins the scene-type registry across four sources —Explainer.tsx's dispatch cases,SCENE_TYPES.md, the schema enum, and_REMOTION_SCENE_TYPES— so a type can never again be routable but unrepresentable. It also asserts every complete artifact example in the director skills validates, which is how therender_reportand documentary-montage examples silently rotted.SCENE_TYPES.md's "adding a new scene type" checklist now includes the schema and the Python set.Reviewer notes
cuts[].typeis a closed enum, not a free string, so typos liketext-cardare still rejected.🤖 Generated with Claude Code