render-editorial-motion-podcast
Assemble an editorial-motion podcast-clip ad from a config — a real clipped podcast MP3 carries the narrative while N flat 2-tone editorial-illustration keyframes are animated NOT by generative i2v but by DETERMINISTIC ffmpeg ken-burns (zoompan) + hard cuts (no crossfades, which
Install
npx skills add https://github.com/gooseworks-ai/goose-skills/tree/main/skills/ads/capabilities/render-editorial-motion-podcast
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install gooseworks-ai-goose-skills@llmmart
git clone https://github.com/gooseworks-ai/goose-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole gooseworks-ai/goose-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
render-editorial-motion-podcast
Assemble an editorial-motion podcast-clip ad from a config: a real clipped podcast audio line carries the whole narrative and every visual beat is timed to the sentence it describes, in a bold flat 2-tone editorial-illustration look ("a New Yorker spot-illustration that moves"). The motion is not generative video but deterministic ffmpeg ken-burns on static keyframes, so it reads as a printed page that moves. This capability is that FREE, deterministic assembly — the ffmpeg motion, hard-concat, audio mux, caption burn, and PIL end card.
scripts/config.example.json is the worked example (Klarify "Rat Park", ~40.8s 1080×1920
9:16, 6 beats); scripts/PIPELINE.md maps every config block to its source step and
scripts/README.md documents the free assembly.
Run
This is the FREE, deterministic assembly stage — it spends nothing on the motion layer.
The paid inputs are separate: the real podcast MP3 is clipped from source (free ffmpeg) with
its Whisper word timings, and one editorial-illustration keyframe per beat (chained ref images
so cage/character geometry holds) comes from create-image-fal (Nano Banana). Given the
clipped audio + words.json + the per-beat keyframes + the real brand wordmark PNG,
render-editorial-motion-podcast renders each keyframe as a ken-burns segment, hard-concats
on the beat, muxes the real audio, burns the mid-sentence captions, and composites the PIL end
card → the master. Re-cuts reuse the existing audio / keyframes and cost $0.
Contract (the free assembly)
- A spoken narration carries the whole spot — no generated SONG. Mux the provided
narration MP3 (
-map 0:v:0 -map 1:a:0) — a real clipped podcast line (preferred) OR an approved generated VO (create-vo-elevenlabs). Never a sung/generated track. (Clip-vs-generate is the recipe's STEP-0 intake decision — if no source episode is supplied, ASK the user.) - NO generative i2v — deterministic ffmpeg ken-burns only. Animate each static keyframe
with
zoompan(push-in / pull-back, 1.0→~1.06×, 24fps); Seedance/Kling are photoreal-trained and invent naturalistic middle states that collapse the 2-tone look. Never-loop 1withzoompan d=N(it balloons the duration); feed a single image and clamp with-t+trim. - Hard cuts on the beat — no crossfades. Crossfades ghost two drifting cages through each other; hard-concat each beat's segments and split long beats into micro-cuts (target 8–10 distinct visual moments). Each beat's visual STARTS within ~0.5s of its spoken line.
- Captions from Whisper word-timestamps, ON only mid-sentence. Burn
frosted-subtlecaptions while the speaker talks; leave silent/reflective beats and the end card uncaptioned. THREE mandatory rules (each bit us in prod — bake them in):- NON-OVERLAP — clamp every line to END before the next STARTS
(
end = min(last_word_end + ~0.15, next_start - 0.03)). Two boxes must never stack at the same spot; an end-tail bleeding into the next window is the #1 caption bug. - SAFE AREA — captions sit in the lower third, so the keyframe's subject must stay in the
upper ~75% (see the recipe's
look_pack.caption_safe_area). If a finished keyframe's subject intrudes into the caption band, deterministically shift the subject UP into the empty top space (PIL: paste up ~0.24H onto a canvas pre-filled with the exact paper color from a clean corner) — never let the box sit on the subject. - BURN ENGINE — prefer libass (
ass/subtitlesfilter), but checkffmpeg -filtersfirst: many builds (Homebrew) lack libass/drawtext. If absent, use the deterministic overlay fallback — render each line as a transparent PNG (frosted rounded box + white text, PIL) and composite via the ffmpegoverlayfilter with timedenable='between(t,st,en)'windows. Same look, no libass.
- NON-OVERLAP — clamp every line to END before the next STARTS
(
- End card via PIL from the real wordmark PNG — never AI-render brand text. The lockup is composited deterministically (stretched-gradient bg + feathered mascot crop + wordmark + tagline with a system font); a diffusion model garbles a wordmark ("therapits"). The video runs a ~1.5s silent hold past the audio on the end card (fade first/last 0.3s).
- FFmpeg composite, deterministic, FREE. Ken-burns each keyframe, hard-concat, mux the real audio, burn the captions, hold on the end card → a 1080×1920 h264+aac master. No paid calls.
Files (goose-skills)
-
scripts
-
config.example.json 14.7 KB
{ "_comment": "Klarify 'Rat Park' editorial-motion podcast-clip ad — the worked example. Copy to config.json and edit. A ~40.8s bold flat 2-tone editorial-illustration (EDITORIAL_2TONE) 9:16 explainer where a REAL podcast audio clip carries the whole narrative (NO generated song, NO AI VO). Pipeline: (1) clip the source episode MP3 to the payoff range + Whisper word timings; (2) one Nano Banana keyframe per beat (9:16, chained ref images off the style anchor, ONE look pack); (3) DETERMINISTIC ffmpeg ken-burns motion + hard-cut concat (NO generative i2v); (4) mux the real audio + frosted-subtle Whisper captions (ON only mid-sentence); (5) PIL brand end card (never AI text). Master 1080x1920, 24fps, h264+aac, ~40.8s. See scripts/PIPELINE.md for the source scripts.", "brand_name": "Klarify", "campaign": "the-rat-park-experiment", "title": "The Rat Park Experiment", "width": 1080, "height": 1920, "fps": 24, "duration_sec": 40.8, "aspect_ratio": "9:16", "audio": { "_comment": "This worked example uses a REAL clipped podcast line (NO generated song). Note the format ALSO supports a generated VO when no source episode is supplied — that's the recipe's STEP-0 intake decision (ASK the user; provider 'generated_vo' via create-vo-elevenlabs from a short truthful script). Here the payoff hinges on host Tamer's real cadence at 'it was more about community.' Clip to the payoff range + a fade-out tail so the trailing 'Because...' doesn't bleed in. THIS AUDIO SETS THE TIMELINE.", "provider": "podcast_clip", "source_mp3": "clients/klarify/podcasts/future-of-therapy/why-motivation-is-the-worst-foundation-for-lasting-change.mp3", "clip_start_sec": 933.5, "clip_end_sec": 973.3, "fade_out": { "st_sec": 39.3, "d_sec": 0.5, "_note": "afade=t=out:st=39.30:d=0.5 — cleans the sharp final 'community.' cut and drops the host's 'Because...' tail" }, "clipped_output": "audio/source-rat-park-40s.mp3", "_clip_cmd": "ffmpeg -y -ss 933.5 -to 973.30 -i <source_mp3> -af 'afade=t=out:st=39.30:d=0.5' -c:a libmp3lame -b:a 192k audio/source-rat-park-40s.mp3 -> 39.84s clip ending cleanly on the payoff line", "transcript": { "_note": "Default to fal-ai/whisper for podcast-length audio (~$0.04, <30s vs. local whisper 40min+ CPU). Word-level timestamps used to mark each beat's payoff word.", "provider": "fal", "model": "whisper", "word_timestamps": true, "output": "audio/words.json" } }, "look_pack": { "_comment": "ONE look pack + CHAINED ref images (pass the anchor + any earlier matching frame as an extra --ref-image) is what makes N beats read as one printed set. Independent Nano Banana rolls drift cage/character geometry; crossfades then expose it, so hard-cut. Reference: moodboard/style-ref-01.png (Niemann/Steinberg/New Yorker lineage).", "name": "EDITORIAL_2TONE", "palette": { "paper": "#F4EBD9", "ink": "#1A1A1A", "accent": "#86987A", "_note": "STRICT 2-tone — cream paper + charcoal ink + ONE sage accent (on the 'COMMUNITY' label only). No rust, no other colors. Nano Banana sometimes adds ambient warmth (a faint orange lamp glow) unprompted — re-roll if it catches the eye." }, "style_ref": "moodboard/style-ref-01.png", "anchor_keyframe": "generated/style-anchor/anchor-cage-1.png", "style_opener": "Editorial illustration: bold flat 2-tone — solid charcoal-black silhouettes on warm cream off-white paper, thick uniform line weight, halftone-dot shading for depth. Christoph Niemann / Saul Steinberg / New Yorker spot-illustration lineage. Printed-magazine editorial illustration, NOT vector-clean, NOT Kurzgesagt, NOT pen-and-marker. 9:16 vertical with cream negative space.", "negative_tail": "No color besides cream and charcoal (plus the single sage accent only where specified). No gradients (halftone dots for shading). No vector cleanliness, no motion-graphics-template look, no photorealism. CRITICAL: ABSOLUTELY NO TEXT, NO LETTERS, NO WORDS, NO LABELS anywhere in the image." }, "keyframe_engine": { "_note": "create-image-nano-banana-fal. Per beat: prompt = look_pack.style_opener + beats[i].keyframe_prompt + look_pack.negative_tail, --ref-image style_ref --ref-image anchor_keyframe (+ any earlier matching frame). Generate the anchor FIRST and approve it before fanning out. ~26 rolls incl. re-rolls ~= $1.00.", "provider": "fal", "model": "nano-banana", "skill": "create-image-nano-banana-fal", "aspect_ratio": "9:16", "chain_refs": true }, "motion_engine": { "_comment": "NO generative i2v. The first pass tried fal-ai/bytedance/seedance/v1/pro/image-to-video and it (a) hallucinated a photoreal human hand holding a marker on the 'pen draws on' beat and (b) morphed the rats into menacing humanoids + hallucinated whimsical cartoon clouds on the decline beat — Seedance is photoreal-trained and invents naturalistic middle states, collapsing the 2-tone discipline. Retired it; saved ~$4-5. Deterministic ffmpeg zoompan on static keyframes = 'printed page that moves' + exact timing control.", "engine": "ffmpeg_kenburns", "fps": 24, "render_scale": "2x_then_downscale", "helpers": { "render_kb": "push-in ken-burns: scale to 2x, crop, zoompan z=min(zoom+step,zoom_end) 1.0->zoom_end (default 1.04), d=dur*fps, trim=duration", "render_kb_out": "pull-back ken-burns: zoompan z from zoom_start (default 1.08) down to 1.0" }, "concat": "hard_cut", "_concat_note": "Hard cuts on the beat — NO crossfades (they ghost two drifting cages through each other; different bar counts / dropper positions). Belt-and-suspenders: NEVER use -loop 1 with zoompan d=N — d runs per input frame per loop, so a 3s target ballooned to 225s. Feed a single image, let zoompan d=<frames> + -t <dur> + trim=duration=<dur> clamp." }, "beats": [ { "id": "beat-1", "role": "HOOK_CAGE1_REVEAL", "t_start": 0.0, "t_end": 3.5, "audio_line": "I remember reading this study, the famous rat study", "payoff_word": "study", "caption_on": true, "keyframe": "generated/style-anchor/anchor-cage-1.png", "keyframe_prompt": "Subject: a small rectangular wire cage with two small rats inside reaching up at a dropper labeled with a tiny test-tube icon. Cream negative space. This is the ANCHOR frame — every later frame chains off it via --ref-image so the cage geometry stays consistent.", "motion": { "type": "render_kb", "dur": 3.5, "zoom_end": 1.05 } }, { "id": "beat-2", "role": "DECLINE_ARC", "t_start": 3.5, "t_end": 11.5, "audio_line": "they laced their water with cocaine ... kept drinking till they died", "payoff_word": "died", "caption_on": true, "panels": [ "generated/beat2-consistent/p-reaching.png", "generated/beat2-consistent/p-sitting.png", "generated/beat2-consistent/p-slumped.png", "generated/beat2-consistent/p4-lying.png" ], "keyframe_prompt": "EXACT SAME cage as the reference image — same bars, same dropper, same charcoal silhouette + halftone style. The ONLY change across the 4 panels is the two rats progressively slumping (reaching -> sitting -> slumped -> lying). USE METAPHOR-FORWARD LANGUAGE ('energy fades / grow tired / settle resting / halftone wisps drift up like quiet breath') — 'cocaine' / 'kept drinking' / 'died' trips FAL's content_policy filter; the metaphor prompt produced the same visual and passed.", "motion": { "type": "render_kb", "dur": 2.0, "per_panel": true, "zoom_end": 1.04, "concat": "hard_cut", "_note": "4 x 2.0s panels, hard-cut concat = 8s" } }, { "id": "beat-3", "role": "REFRAME_RATPARK", "t_start": 11.5, "t_end": 22.0, "audio_line": "for so many years people were like oh my god ... they redid that study ... they gave it a really nice cage", "payoff_word": "really nice cage", "caption_on": true, "_note": "Split into 3 sub-beats so each aligns to a DIFFERENT sentence (LEARNINGS L3): 12-15 'for so many years' -> dead-rats reflective hold; 15-19 'they redid the study, tiny cage' -> empty new cage; 19-22 'a really nice cage' -> populated Rat Park lands exactly on its line. Concat sub-segments hard-cut. Total 10s.", "sub_segments": [ { "id": "b3a-dead-hold", "dur": 3.0, "audio_anchor": "for so many years (reflective hold)", "keyframe": "generated/beat2-consistent/p4-lying.png", "motion": { "type": "render_kb_out", "zoom_start": 1.06 } }, { "id": "b3b-empty", "dur": 4.0, "audio_anchor": "they redid that study, instead of a tiny cage", "keyframe": "generated/ratpark-empty.png", "motion": { "type": "render_kb", "zoom_end": 1.05 } }, { "id": "b3c-full", "dur": 3.0, "audio_anchor": "a really nice cage (populated Rat Park revealed)", "keyframe": "generated/style-anchor/anchor-cage-2-rat-park.png", "motion": { "type": "render_kb", "zoom_end": 1.04 } } ], "motion": { "concat": "hard_cut" } }, { "id": "beat-4", "role": "PULLBACK_COMMUNITY", "t_start": 22.0, "t_end": 30.0, "audio_line": "bunch of fun things to do and other friends and lo and behold, didn't touch the cocaine-laced water", "payoff_word": "didn't touch", "caption_on": true, "keyframe": "generated/beat4-wide-v3.png", "keyframe_prompt": "Wide pull-back — both cages side-by-side. A 'COMMUNITY' label punches in — the ONLY sage-accent element in the whole spot. NOTE: the wide pull-back kept generating tiny 'CAGE 1' labels even with 'NO TEXT'; the all-caps 'ABSOLUTELY NO TEXT, NO LETTERS, NO WORDS, NO LABELS' tail fixed it. Composite the deliberate COMMUNITY label separately (PIL/drawtext) if Nano Banana can't be trusted with it.", "motion": { "type": "render_kb", "dur": 8.0, "zoom_end": 1.05, "_note": "pull-back zoompan + render_kb on the COMMUNITY reveal" } }, { "id": "beat-5", "role": "PIVOT_THERAPISTS", "t_start": 30.0, "t_end": 37.0, "audio_line": "mind-blowing. It's very true. And it wasn't just about fun.", "payoff_word": "community", "caption_on": true, "_note": "THE PIVOT (the conceptual fix, LEARNINGS L11): the cages dissolve rats into therapists — solo therapist hunched over a laptop at midnight (Cage 1) vs. supported therapists + a Klarify glow on a closed laptop (Cage 2). Split into 3 micro-cuts (full split 3s -> solo CU 1.5s -> supported CU 2.5s) = 3 visual moments in one beat so it doesn't read static (LEARNINGS L10). This sells Klarify to therapists, not therapy to clients.", "sub_segments": [ { "id": "b5-split", "dur": 3.0, "keyframe": "generated/beat5-split.png", "motion": { "type": "render_kb", "zoom_end": 1.04 } }, { "id": "b5-solo-cu", "dur": 1.5, "keyframe": "generated/beat5-solo-closeup.png", "motion": { "type": "render_kb", "zoom_end": 1.05 } }, { "id": "b5-supported-cu", "dur": 2.5, "keyframe": "generated/beat5-supported-closeup.png", "motion": { "type": "render_kb", "zoom_end": 1.05 } } ], "motion": { "concat": "hard_cut" } }, { "id": "beat-6", "role": "END_CARD", "t_start": 37.0, "t_end": 40.8, "audio_line": "It was more about community. (+ ~1.5s silent hold)", "payoff_word": "", "caption_on": false, "keyframe": "generated/klarify-endcard-9x16-final.png", "motion": { "type": "kenburns_static", "dur": 4.3, "zoom_end": 1.03, "fade_in": 0.3, "fade_out": 0.3, "_note": "PIL-composited end card; ~2.8s active + ~1.5s silent hold. The video runs past the audio for the hold (fade the first/last 0.3s so it doesn't hard-black-flash)." } } ], "captions": { "_comment": "Whisper (fal) on the MASTER -> frosted-subtle ASS burn. ON only while the speaker is mid-sentence — leave silent/reflective beats (and the end card) uncaptioned so captions don't fight the visual. This is REAL spoken VO, so Whisper works (unlike sung audio, which returns 'Music Playing'). ls the burn-in-captions skill's templates/ before assuming a style exists — only frosted-subtle + karaoke-fill were present (clean-bubble was documented but missing).", "provider": "fal", "model": "whisper", "style": "frosted-subtle", "skill": "burn-in-captions", "on_when": "speaker_mid_sentence_only", "no_overlap": "MANDATORY — clamp each line to end before the next starts (end = min(last_word_end+~0.15, next_start-0.03)); two boxes must never stack. The #1 caption bug.", "safe_area": "Captions sit in the lower third; keep the subject in the upper ~75%. If a keyframe subject intrudes into the caption band, shift the subject UP into the empty top space (PIL) before assembly.", "burn_engine": "Prefer libass; if `ffmpeg -filters` lacks it (common on Homebrew), render each line as a transparent PNG (frosted box + text) and composite via the `overlay` filter with timed enable windows." }, "end_card": { "_comment": "Brand lockup composited via PIL from the REAL brand asset (the canonical Klarify wordmark PNG) — NEVER AI-rendered text (Nano Banana spelled 'AI to empower therapists' as 'therapits'). Before generating an end card from scratch, `find <brand> -name '*wordmark*'` — the brand team already produced the right asset.", "engine": "pil", "script": "build_endcard.py", "brand_asset": "clients/klarify/ad-runs/future-of-therapy-practice-stakes-low/brand-assets/logos/klarify-wordmark-on-green.png", "wordmark_text": "klarify", "wordmark_font": "/System/Library/Fonts/Helvetica.ttc", "tagline_text": "AI to empower therapists, not replace them", "tagline_font": "/System/Library/Fonts/Helvetica.ttc", "background": "stretch a 1px-wide center column of the source PNG to 1080x1920, GaussianBlur(80) to kill banding", "mascot_crop": "feathered radial-alpha ellipse crop of the brand 'carrot' mascot from the source PNG (box 1290,700,1750,1200), resized to ~30% width, pasted centered-upper (cy = 0.30*H)", "overlay_window_sec": [37.0, 40.8] }, "audio_mix": { "_note": "The real podcast VO carries the spot. Any bed is soft licensed acoustic (finger-picked guitar / soft strings, swell on 'community') at <= -22 LUFS ducked under the VO — NEVER a generated song. Pencil-scribble SFX per drawn line, light not cartoony.", "loudness_lufs": -22, "vo_source": "real_podcast_clip", "music_bed": "optional_licensed_only", "duck_under_vo": true }, "post_production": { "music": { "default": "off", "note": "the real podcast VO carries the spot; a bed is optional licensed acoustic <= -22 LUFS ducked under the VO, never generated" }, "captions": { "default": "on", "note": "frosted-subtle Whisper captions, ON only while the speaker is mid-sentence (not silent beats)" }, "end_card": { "default": "on", "note": "PIL end card from the real brand wordmark PNG, default on" } } } -
PIPELINE.md 6.6 KB
# Pipeline — editorial-motion-podcast How `config.example.json` maps to the real production steps. This molecule ships a **config + this map**, not a bundled runner: the worked example (Klarify "Rat Park") was produced by hand-run bash + PIL scripts that live in `clients/klarify/ad-runs/future-of-therapy-rat-park/scripts/`, with the full reproducible recipe in that run's `how-to.md` and the gotchas in its `learnings.md` (L1–L13). The steps run **in order** because each depends on the last: the clipped audio sets the timeline, the timeline marks the beats, the style anchor locks the look pack, the anchor seeds every keyframe, the keyframes drive the ffmpeg motion, and assembly muxes the real audio + Whisper captions + PIL end card. **Total cost ≈ $3.40** end-to-end (or ~$1.78 if you skip the retired Seedance detour) — the motion + assembly + end card layers are **$0**. ## 1. Clip audio + transcribe (config: `audio`) [cheap ~$0.04] **Clip the real podcast to the payoff moment — it sets the timeline.** Find the moment in the source transcript and `ffmpeg -ss/-to` the episode MP3 to `audio.clip_start_sec .. clip_end_sec`, adding `afade=t=out` (`audio.fade_out`) so the trailing next sentence doesn't bleed in: ```bash ffmpeg -y -ss 933.5 -to 973.30 -i <source_mp3> \ -af "afade=t=out:st=39.30:d=0.5" -c:a libmp3lame -b:a 192k \ audio/source-rat-park-40s.mp3 # -> 39.84s clip ending cleanly on "community." ``` Then Whisper word timings via the **`transcribe-audio-fal`** atom (`fal-ai/whisper`, ~$0.04, <30s — default to it; local whisper ran 40min+ CPU and was killed, L8) → `audio/words.json`. Mark every payoff word's time; each beat's visual must START within ~0.5s of its line (never ahead = spoils the reveal, never lag = redundant, L3). There is **no generated song and no VO** — the host's real recorded line is the narration. ## 2. Style anchor → Nano Banana (config: `look_pack.style_ref`, `look_pack.anchor_keyframe`, `keyframe_engine`) [PAID] `create-image-nano-banana-fal` from `look_pack.style_ref` (`moodboard/style-ref-01.png`) → the FIRST keyframe (`generated/style-anchor/anchor-cage-1.png`, "Cage 1"). This anchor is the reference EVERY subsequent keyframe chains off (`--ref-image`), so cage/character geometry stays consistent. **Approve the anchor before fanning out** — it locks the whole look pack (palette, line weight, halftone discipline, geometry). ## 3. Remaining keyframes → Nano Banana (chained refs!) (config: `beats[].keyframe(s)`, `look_pack`, `keyframe_engine`) [PAID] Per beat, build the prompt as `look_pack.style_opener + beats[i].keyframe_prompt + look_pack.negative_tail`, and call Nano Banana **passing the anchor (and any earlier matching frame) as an additional `--ref-image`** — otherwise Nano Banana invents a new cage every time (L2). One PNG per beat (or a small panel set for the decline arc). Full set ≈ 26 rolls incl. re-rolls (~$1.00). Keep prompts **metaphor-forward, never clinical** (`cocaine` / `kept drinking till they died` trips FAL's `content_policy` filter; "energy fades / settle resting" gives the same visual and passes, L6). Review ALL N before motion; re-roll drift, leaked color, or a leaked "CAGE 1" label (add all-caps "ABSOLUTELY NO TEXT, NO LETTERS, NO LABELS", L-misc). ## 4. ffmpeg ken-burns motion + assembly → `build_v4.sh` (config: `beats[].motion`, `motion_engine`) [FREE] `clients/klarify/ad-runs/future-of-therapy-rat-park/scripts/build_v4.sh` renders each keyframe as a ken-burns segment and hard-concats them. **NO generative i2v** — Seedance hallucinated a photoreal drawing hand + morphed rats into humanoids + hallucinated cartoon clouds; it's photoreal-trained and can't hold the 2-tone discipline (L1, retired). The two reusable helpers: - `render_kb <src> <out> <dur> [zoom_end=1.04]` — push-in: scale to 2×, crop, `zoompan z='min(zoom+step,zoom_end)' d=dur*fps`, `trim=duration=dur`, `-t dur`. - `render_kb_out <src> <out> <dur> [zoom_start=1.08]` — pull-back: `zoompan` from `zoom_start` down to `1.0` (reflective held frames, e.g. the dead-rats hold). Long beats split into micro-cuts (target **8–10 distinct visual moments**; anything held > 6s reads static even with active zoompan, L10). **Hard-concat on the beat — NO crossfades** (they ghost two drifting cages, L2). **Never `-loop 1` with `zoompan d=N`** — d runs per input frame per loop, so a 3s target ballooned to 225s (L7); feed a single image, clamp with `-t` + `trim=duration`. Concat all beats → `edits/master-video-only.mp4`, then mux the real clip letting the video run a ~1.5s silent hold past the audio: ```bash ffmpeg -y -i edits/master-video-only.mp4 -i audio/source-rat-park-40s.mp3 \ -c:v copy -c:a aac -b:a 192k -map 0:v:0 -map 1:a:0 edits/master.mp4 ``` ## 5. End card → `build_endcard.py` (config: `end_card`) [FREE] `scripts/build_endcard.py` composites the brand lockup via **PIL** from the REAL brand wordmark PNG (`end_card.brand_asset`): stretch a 1px center column of the source → 1080×1920 + `GaussianBlur(80)` for the gradient bg; feathered radial-alpha ellipse crop of the brand mascot pasted centered-upper; wordmark (Helvetica 160) + tagline (Helvetica 48) drawn with `ImageDraw.text` → `generated/klarify-endcard-9x16-final.png`. **Never AI-render brand text** — Nano Banana spelled "therapists" as "therapits" (L4). Before building from scratch, `find <brand> -name "*wordmark*"` — the brand team already made the right asset (L9). ## 6. Captions + final (config: `captions`) [PAID ~$0.04] Whisper (fal) on `edits/master.mp4` → `frosted-subtle` ASS via the **`burn-in-captions`** skill → `edits/master-captioned.mp4` (the FINAL deliverable). Captions **ON only while the speaker is mid-sentence** — leave silent/reflective beats and the end card uncaptioned so they don't double up with the visual. This is real spoken VO, so Whisper works (sung audio returns "🎵 Music Playing 🎵"). `ls templates/` before assuming a style exists — only `frosted-subtle` + `karaoke-fill` were present. ```bash python3 .claude/skills/burn-in-captions/scripts/caption.py \ --video edits/master.mp4 --style frosted-subtle cp edits/master.captions/master__frosted-subtle__ass.mp4 edits/master-captioned.mp4 ``` ## 7. Watch / QC [~$0] `/watch:watch edits/master-captioned.mp4 --fps 2` and Read every 4–5 frames across the full duration — the ONLY honest loop (single frames miss timing drift, caption collisions, black first-frame, band artifacts, L5). Iterate on timing per findings. --- Re-cuts (new beat windows, added micro-cuts, re-timed caption ranges, swapped end card) reuse the existing clipped audio / keyframes / clips and cost **$0** — only steps 1–3 (+ the two Whisper calls) spend. -
README.md 4.5 KB
# render-editorial-motion-podcast scripts — the FREE assembly `render-editorial-motion-podcast` is the **deterministic, $0 assembly stage** of the editorial-motion podcast-clip format. The paid stages — the two Whisper calls (~$0.04 each) and the per-beat Nano Banana keyframes — are upstream (`create-image-fal` + a transcription step); the real podcast MP3 is clipped from source with free ffmpeg. This capability spends nothing on the motion layer: it takes the clipped audio + `words.json` + the per-beat keyframes + the real brand wordmark PNG and stitches the finished master. Re-cuts (new beat windows, added micro-cuts, re-timed caption ranges, a swapped end card) reuse the existing audio / keyframes and cost **$0**. `config.example.json` is the worked example (Klarify "Rat Park", ~40.8s 1080×1920). `PIPELINE.md` maps every config block to its source step. This README documents the FREE assembly pieces that `render-editorial-motion-podcast` owns. ## 1. ffmpeg ken-burns motion — NOT generative i2v The motion is **deterministic ffmpeg ken-burns** on static keyframes, never generative i2v. Seedance/Kling are photoreal-trained and invent naturalistic middle states (a photoreal drawing hand, rats morphing into humanoids, hallucinated clouds) that collapse the 2-tone discipline. Two helpers: `render_kb <src> <out> <dur> [zoom_end]` (push-in — scale to 2×, crop, `zoompan z='min(zoom+step,zoom_end)'`, `trim=duration`, `-t dur`) and `render_kb_out` (pull-back, `zoom_start` down to 1.0). **Never** `-loop 1` with `zoompan d=N` — d runs per input frame per loop and balloons the duration; feed a single image and clamp with `-t` + `trim`. ## 2. Hard-concat on the beat — no crossfades Each beat's segments are hard-concatenated on the beat; long beats split into micro-cuts (target 8–10 distinct visual moments — anything held > 6s reads static even with active zoompan). **No crossfades** — they ghost two drifting cages through each other. Each beat's visual STARTS within ~0.5s of its spoken line (marked from the Whisper timings upstream), never ahead, never lag. ## 3. Narration mux + mid-sentence captions The narration MP3 is muxed as the master audio (`-map 0:v:0 -map 1:a:0`) — no generated song. The narration is a real clipped podcast line (preferred) OR an approved generated VO (`create-vo-elevenlabs`); clip-vs-generate is the recipe's STEP-0 intake decision (if no source episode is supplied, ASK the user — don't fabricate or silently skip). The video runs a ~1.5s silent hold past the audio on the end card (fade the first/last 0.3s so it doesn't hard-black-flash). Captions are `frosted-subtle` (Whisper word-timestamps on the master) **ON only while the speaker is mid-sentence** — silent/reflective beats and the end card stay uncaptioned. **Three mandatory rules, all learned the hard way:** - **NON-OVERLAP** — clamp every line to END before the next STARTS (`end = min(last_word_end + ~0.15, next_start - 0.03)`). Two boxes must never be on screen at once; an end-tail bleeding into the next window stacks two boxes at the same spot (the #1 caption bug). Verify zero remaining window overlaps before burning. - **SAFE AREA** — captions sit in the lower third, so the keyframe's subject must live in the upper ~75% (recipe `look_pack.caption_safe_area`). If a finished keyframe's subject intrudes into the caption band, deterministically **shift the subject up** into the empty top space (PIL: paste up ~0.24H onto a canvas pre-filled with the exact paper color sampled from a clean corner, so the vacated bottom is clean paper) — never let the box sit on the subject. - **BURN ENGINE** — prefer libass (`ass`/`subtitles`), but **check `ffmpeg -filters` first**: many builds (Homebrew) ship without libass/drawtext. If absent, use the deterministic **overlay fallback** — render each caption line as a transparent PNG (frosted rounded box + white text, PIL) and composite with the ffmpeg `overlay` filter using timed `enable='between(t,st,en)'` windows. Identical frosted-subtle look, no libass required. ## 4. PIL end card — from the real wordmark PNG, no AI text The brand lockup is composited with **PIL** from the brand's REAL wordmark PNG: stretch a 1px center column → 1080×1920 + `GaussianBlur(80)` for the gradient bg; a feathered radial-alpha ellipse crop of the brand mascot pasted centered-upper; wordmark + tagline drawn with a system font. The brand text is **never** AI-rendered — a diffusion model garbles a wordmark ("therapits"). Output is a 1080×1920 h264 + aac master. Deterministic, no paid calls, no keys.
-
-
tests
-
smoke-test.md 1.1 KB
# Smoke Test Given the clipped REAL podcast audio, one 2-tone editorial keyframe per beat, the audio word timings, and the brand's real wordmark, `render-editorial-motion-podcast` assembles the master: ffmpeg ken-burns each keyframe to its beat window, hard-concat on the beat, burn frosted-subtle captions, composite the PIL end card, mux the real audio → 1080×1920 h264+aac (~40s). Pass when the assembly runs to a valid MP4 and: - motion is DETERMINISTIC ffmpeg ken-burns only (NO generative i2v — it breaks the 2-tone look); - each payoff visual starts within ~0.5s of its spoken line; beats hard-cut (no crossfades); - the one look pack + strict 2-tone palette hold across all beats (no leaked color or text); - captions are ON only mid-sentence; the end card is composited via PIL from the real wordmark (no AI-rendered brand text); - the REAL podcast audio carries with no generated song and no VO; - **no paid call is made** — the keyframes come from the paid capability (create-image-fal); this assembly is $0 and a re-cut reuses the existing audio/keyframes.
-
-
SKILL.md 5.2 KB
--- name: render-editorial-motion-podcast description: Assemble an editorial-motion podcast-clip ad from a config — a real clipped podcast MP3 carries the narrative while N flat 2-tone editorial-illustration keyframes are animated NOT by generative i2v but by DETERMINISTIC ffmpeg ken-burns (zoompan) + hard cuts (no crossfades, which expose geometric drift), each beat snapped to its spoken line, the real audio muxed, Whisper-driven captions burned only mid-sentence, and closed on a PIL brand end card — never AI-rendered text. This is the FREE deterministic assembly stage (ffmpeg ken-burns + hard concat + audio mux + captions + end card); the real audio is clipped from source and the keyframes come from create-image-fal. Use for the editorial-motion-podcast format. status: active --- # render-editorial-motion-podcast Assemble an **editorial-motion podcast-clip** ad from a config: a real clipped podcast audio line carries the whole narrative and every visual beat is timed to the sentence it describes, in a bold flat 2-tone editorial-illustration look ("a New Yorker spot-illustration that moves"). The motion is **not generative video** but deterministic ffmpeg ken-burns on static keyframes, so it reads as a printed page that moves. This capability is that **FREE, deterministic assembly** — the ffmpeg motion, hard-concat, audio mux, caption burn, and PIL end card. `scripts/config.example.json` is the worked example (Klarify "Rat Park", ~40.8s 1080×1920 9:16, 6 beats); `scripts/PIPELINE.md` maps every config block to its source step and `scripts/README.md` documents the free assembly. ## Run This is the **FREE, deterministic** assembly stage — it spends nothing on the motion layer. The paid inputs are separate: the real podcast MP3 is clipped from source (free ffmpeg) with its Whisper word timings, and one editorial-illustration keyframe per beat (chained ref images so cage/character geometry holds) comes from `create-image-fal` (Nano Banana). Given the clipped audio + `words.json` + the per-beat keyframes + the real brand wordmark PNG, `render-editorial-motion-podcast` renders each keyframe as a ken-burns segment, hard-concats on the beat, muxes the real audio, burns the mid-sentence captions, and composites the PIL end card → the master. Re-cuts reuse the existing audio / keyframes and cost **$0**. ## Contract (the free assembly) - **A spoken narration carries the whole spot — no generated SONG.** Mux the provided narration MP3 (`-map 0:v:0 -map 1:a:0`) — a real clipped podcast line (preferred) OR an approved generated VO (`create-vo-elevenlabs`). Never a sung/generated track. (Clip-vs-generate is the recipe's STEP-0 intake decision — if no source episode is supplied, ASK the user.) - **NO generative i2v — deterministic ffmpeg ken-burns only.** Animate each static keyframe with `zoompan` (push-in / pull-back, 1.0→~1.06×, 24fps); Seedance/Kling are photoreal-trained and invent naturalistic middle states that collapse the 2-tone look. Never `-loop 1` with `zoompan d=N` (it balloons the duration); feed a single image and clamp with `-t` + `trim`. - **Hard cuts on the beat — no crossfades.** Crossfades ghost two drifting cages through each other; hard-concat each beat's segments and split long beats into micro-cuts (target 8–10 distinct visual moments). Each beat's visual STARTS within ~0.5s of its spoken line. - **Captions from Whisper word-timestamps, ON only mid-sentence.** Burn `frosted-subtle` captions while the speaker talks; leave silent/reflective beats and the end card uncaptioned. THREE mandatory rules (each bit us in prod — bake them in): 1. **NON-OVERLAP** — clamp every line to END before the next STARTS (`end = min(last_word_end + ~0.15, next_start - 0.03)`). Two boxes must never stack at the same spot; an end-tail bleeding into the next window is the #1 caption bug. 2. **SAFE AREA** — captions sit in the lower third, so the keyframe's subject must stay in the upper ~75% (see the recipe's `look_pack.caption_safe_area`). If a finished keyframe's subject intrudes into the caption band, deterministically **shift the subject UP** into the empty top space (PIL: paste up ~0.24H onto a canvas pre-filled with the exact paper color from a clean corner) — never let the box sit on the subject. 3. **BURN ENGINE** — prefer libass (`ass`/`subtitles` filter), but **check `ffmpeg -filters` first**: many builds (Homebrew) lack libass/drawtext. If absent, use the deterministic **overlay fallback** — render each line as a transparent PNG (frosted rounded box + white text, PIL) and composite via the ffmpeg `overlay` filter with timed `enable='between(t,st,en)'` windows. Same look, no libass. - **End card via PIL from the real wordmark PNG — never AI-render brand text.** The lockup is composited deterministically (stretched-gradient bg + feathered mascot crop + wordmark + tagline with a system font); a diffusion model garbles a wordmark ("therapits"). The video runs a ~1.5s silent hold past the audio on the end card (fade first/last 0.3s). - **FFmpeg composite, deterministic, FREE.** Ken-burns each keyframe, hard-concat, mux the real audio, burn the captions, hold on the end card → a 1080×1920 h264+aac master. No paid calls. -
skill.meta.json 339 B
{ "slug": "render-editorial-motion-podcast", "category": "capabilities", "domain": "ads", "tags": [ "ads" ], "installation": { "base_command": "npx goose-skills install render-editorial-motion-podcast", "supports": [ "claude", "cursor", "codex" ] }, "requires_skills": [ "watch" ] }
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.