Commit 3264de1
docs: regenerate creative-pipeline media with gpt-image-2
flux/schnell was chosen on cost — $0.003 per megapixel — and it was the wrong
trade. It mangles hands, and this brief requires them in three of four shots. A
model that cannot draw the thing being commissioned is not cheap. Stills now
come from openai/gpt-image-2; clips stay on LTX-distilled, which was never the
problem. Both are overridable with CREATIVE_IMAGE_MODEL / CREATIVE_VIDEO_MODEL.
Regenerated from the prompts recorded in the run dump rather than by re-running
the agent, so this cost image and video calls and not another loop.
The swap introduced a different brief violation, which is the more interesting
half. flux could not render legible text, so it never breached the brief's
no-logo rule by accident. gpt-image-2 can, and branded the bottle in shot 3
with raised COLD BREW COFFEE lettering. Better capability, new failure mode.
The prompt is why: sonnet never carried no-logo into any image prompt. It read
the brief and the constraint did not survive into the generation call. Shot 3
was regenerated with it appended.
Neither violation is visible to the suite. forbiddenAvoided reads shot action
text and layer names, so it can see the word logo in a prompt and never what
the picture contains; mustFeature asks only whether each element appears
somewhere in the shot list, so a shot with neither hands nor sunrise passes.
The predicates grade the plan. Grading the artifact needs a human or a vision
model, and that is now written down in both the fixture README and the package
docs rather than left for someone to discover.
Co-Authored-By: Claude <noreply@anthropic.com>1 parent 0256488 commit 3264de1
12 files changed
Lines changed: 62 additions & 24 deletions
File tree
- docs/evals/creative-pipeline
- packages/agents
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
29 | 32 | | |
30 | | - | |
31 | | - | |
| 33 | + | |
| 34 | + | |
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| |||
45 | 48 | | |
46 | 49 | | |
47 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
48 | 70 | | |
49 | 71 | | |
50 | 72 | | |
51 | 73 | | |
52 | 74 | | |
53 | 75 | | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading
Loading
Loading
Loading
Loading
0 commit comments