Open build log · help wanted
We built an AI short-film pipeline with Codex + Claude Code — and it's been stuck for ~20 days
This is a request for advice, not a tutorial. Real project, real blockers, real numbers — all public. If you actually know image-to-video realism or long-horizon agent workflows, we'd love to be wrong.
20 days, stuck in the same place. This time the one in the pit is us.
What we're building
An (aspirationally) automated pipeline for faux-documentary shorts. First film: "2:17 AM — there's an employee in my store who doesn't exist." Through CCTV, shaky phone clips and chat logs, a tiny convenience-store owner discovers a fictional AI employee, "Xia," who's been replying to bad reviews and re-cutting his short videos at night. Original virtual character — no real faces, no face-swap. Spec: 110–130s, 9:16, sixteen 5–9s shots, a new beat every 8–12s, hook in the first 3 seconds.
The framework (this part is honestly solid)
Built by Codex + Claude Code working together, in stages:
script → shotlist + gen-prompts → character lookbook → image-to-video (16 shots) → audio → edit → platform test
- Kling — character consistency, lip-sync, multi-shot
- Runway Gen-4 — realistic motion, phone/CCTV texture
- HyperFrames / Remotion — captions, UI evidence, compositing
- Homegrown Python — asset acceptance, readiness gates, external-model review loop
We even gave it hard acceptance gates to stop self-deception: a pipeline:gate that only renders when
render_allowed=true; a rubric we send to Grok/Gemini/Claude (first-3s hold, event-driven, believability…) where
overall < 7.6 can't go to platform test; and a rule that "technically valid ≠ content valid." That process
discipline is, frankly, stricter than most people doing AI video.
Then we got stuck — on two levels
Footage can't pass the realism bar
We want the rough realism of convenience-store CCTV / phone candid — cold fluorescent light, compression noise, vertical handheld shake. Image-to-video keeps coming out either too "AI showreel / ad studio," or with an unstable face. Photoreal believability never clears our own ≥ 7.5 line; character consistency drifts across shots.
The agent fell into over-engineering paralysis
Because the gate is stuck on "assets not landed," the collaborating agent's only daily move is read-only inspection plus writing reflections: watchlist → delta → streak → trend → scaffold → sync, looping. So it keeps building more tools — landing watchers, cross-day trend summaries, a path-regression verifier, endless readiness re-checkers. Every step locally reasonable; the sum is a treadmill.
The number that says it all
Our video pipeline has not produced a single finished clip yet — and tools/ already holds 437 Python scripts. The agent is extremely good at making the process more airtight. So airtight it forgot the goal was to ship a film.
Questions for anyone who actually knows
- How do you reliably get "CCTV / found-footage realism" out of image-to-video? Prompting, wrong model, or post-process noise/degrade/shake?
- Character consistency across 16 shots in 2026 — is Kling's character reference enough, or train a LoRA / use something else?
- The one we most want answered: when you run long-horizon tasks with Codex / Claude Code / Cursor, how do you stop the agent from over-engineering — building process instead of shipping?
What we've already tried (not lazy)
- Heavy negative prompts to kill the "AI look" (cartoon / sci-fi / beauty filter / distorted face…)
- An external-model review loop with a quantified rubric and hard thresholds
- Decomposing into "stills first → then i2v → one action per shot" to reduce complexity
- Adding fail-close gates to the agent — which is exactly what fed the over-engineering problem
Reach out
Full pre-production pack, shot prompts, the review rubric, and the structure of those 437 scripts — open to anyone who wants a real exchange. This is a genuine episode of "AI tools, the hard way" — except this time the one who fell in the pit is us.
Open build log · published as-is · we will update this page when we get unstuck.