Claude Skill

render-song-mv

Assemble a song-driven music-video ad from a config — a generated sung track carries the whole narration across N tableaux (one keyframe -> one i2v clip per lyric beat) with NO separate voiceover, captions synced to the song's OWN word timings (script-window, never Whisper) and t

LLM Mart · 0 points · 6 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download gooseworks-ai-goose-skills-skills_ads_capabilities_render-song-mv-e1592ee.zip · 12 KB
Part of gooseworks-ai/goose-skills — 44 skills

Install

skills CLI npx skills add https://github.com/gooseworks-ai/goose-skills/tree/main/skills/ads/capabilities/render-song-mv
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install gooseworks-ai-goose-skills@llmmart
Git git clone https://github.com/gooseworks-ai/goose-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole gooseworks-ai/goose-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

render-song-mv

Assemble a song-driven music-video ad from a config: a purpose-written, sung song is the entire narration (no separate voiceover), and every visual beat is timed to the lyrics. The delivered song sets the timeline; N tableaux (one keyframe → one image-to-video clip per lyric beat, all in a single look pack) are cut to their lyric windows and hard-concatenated on the beat, captions are built from the song's OWN word timings with the hook line landing on the chorus drop, and the spot closes on a PIL brand end card. It reads like a tiny animated music video, not a demo. scripts/config.example.json is the worked example (Loóna "Fall In Love With Sleep Again", 28s paper-craft 9:16); scripts/PIPELINE.md maps every config block to its step and scripts/README.md documents the free assembly.

Run

This is the FREE, deterministic assembly stage — it spends nothing. The three paid inputs are separate capabilities: the sung song (create-music-elevenlabs, music_v1, force_instrumental FALSE — the lyrics ARE the script, returns mp3 + words.json), one keyframe per tableau (create-image-fal), and one Kling 3.0 i2v clip per tableau (create-video-fal). Given the delivered song + words.json + one clip per beat, render-song-mv cuts each clip to its lyric window, hard-concats on the beat, builds the lyric-synced captions, composites the PIL end card, and muxes → the master. Re-cuts reuse the existing song / keyframes / clips and cost $0.

Contract (the free assembly)

  • The sung song carries the narration — no separate VO. The generated ElevenLabs track IS the bed and the script (force_instrumental false); do not add a spoken voiceover or a second music bed.
  • Plan the timeline AROUND the delivered song. The song is generated first and reshapes/ overshoots length; snap every tableau boundary to the lyric-phrase edges in the returned word timings (timeline.json) — never trim the song to a pre-planned grid.
  • Captions from the song's OWN word timings, not Whisper (script-window). Chunk audio/words.json (~3 words at lyric boundaries); accent words get the warm-glow color. Whisper on sung audio returns "🎵 Music Playing 🎵", so it can't caption lyrics.
  • Land the hook on the chorus drop. Exactly ONE hero tableau (is_hook) is timed so the payoff word (song.hook_word) sits on the chorus drop; accent that word in the captions.
  • One look pack for consistency. A single style_opener + negative_tail + palette drives every keyframe so N beats read as one film; no morph within a clip.
  • Hard cuts on the beat. Cut each clip to its lyric window and hard-concat — no dissolves (one optional match-cut into the hero reveal).
  • PIL end card from the real app icon — never AI-render brand text. The lockup is composited deterministically (brand gradient + circular app icon + wordmark + tagline + CTA) from the brand's real asset; a diffusion model garbles a wordmark.
  • FFmpeg composite, deterministic, FREE. Burn the caption ASS, overlay the end-card PNG on the final window, mux the song, boost the climax beat, loudnorm to −14 LUFS → 1080×1920 h264+aac. No paid calls, no keys.
Files (goose-skills)
  • scripts
    • config.example.json 15.7 KB
      {
        "_comment": "Lo\u00f3na 'Fall In Love With Sleep Again' song-driven music-video \u2014 the worked example. Copy to config.json and edit. A ~28s paper-craft (DREAMSCAPE_NIGHT) 9:16 music video where a GENERATED SUNG SONG carries the whole narrative (NO separate VO). Pipeline: (1) ElevenLabs music_v1 song from song.structure -> mp3 + words.json; (2) one Higgsfield gpt_image_2 keyframe per tableau (2k 9:16, one look_pack); (3) one Higgsfield kling3_0 i2v clip per tableau; (4) captions from the song's WORD timings, hook word landing on the chorus drop; (5) hard-cut assembly + PIL brand end card + song mux, -14 LUFS. Master 1080x1920 (keyframes rendered at 2k ~= 1520x2688 then downscaled in assembly). See scripts/PIPELINE.md for the source scripts.",
        "brand_name": "Lo\u00f3na",
        "campaign": "fall in love with sleep again",
        "title": "Fall In Love With Sleep Again",
        "width": 1080,
        "height": 1920,
        "fps": 30,
        "duration_sec": 28,
        "keyframe_resolution": "2k",
        "aspect_ratio": "9:16",
        "song": {
          "_comment": "Generated FIRST. The song carries the narration \u2014 there is NO separate voiceover. The delivered track sets the timeline; snap every tableau boundary to its lyric window (see the run's working/timeline.json). No artist names in the prompt (ElevenLabs Music ToS filter).",
          "provider": "elevenlabs",
          "model": "music_v1",
          "lyrics_md": "source/lyrics-locked.md",
          "bpm": 80,
          "prompt": "Dreamy synthwave lullaby with a soft breathy female lead vocal. Atmospheric reverb pads, gentle music-box chime arpeggios, soft finger-snap kick, warm low-end. Verse: piano + vocal whisper. Pre-chorus: pads build, soft strings enter. Chorus: full atmospheric drop, vocal floats over moonlit synths, hook lands on the word 'fall'. Outro: stripped back to music-box and breath.",
          "force_instrumental": false,
          "hook_line": "fall in love with sleep again",
          "hook_word": "fall",
          "hook_target_sec": 15.24,
          "structure": [
            {
              "section": "Intro",
              "duration_ms": 3000,
              "lines": []
            },
            {
              "section": "Verse",
              "duration_ms": 7000,
              "lines": [
                "two AM and the screen still glows",
                "endless feeds and the night just slows"
              ]
            },
            {
              "section": "Pre-Chorus",
              "duration_ms": 5000,
              "lines": [
                "then i tap into something soft",
                "all the noise of the day falls off"
              ]
            },
            {
              "section": "Chorus",
              "duration_ms": 8000,
              "lines": [
                "fall in love with sleep again",
                "color worlds till the dreaming ends",
                "loo-nah, loo-nah, take me in"
              ]
            },
            {
              "section": "Outro",
              "duration_ms": 5000,
              "lines": [
                "fall in love with sleep again"
              ]
            }
          ],
          "outputs": {
            "audio": "audio/music.mp3",
            "word_timings": "audio/words.json",
            "metadata": "audio/music_metadata.json"
          }
        },
        "look_pack": {
          "name": "DREAMSCAPE_NIGHT",
          "palette_anchors": [
            "lavender night",
            "soft moonlight silver",
            "dusty pink",
            "mint",
            "cream paper",
            "deep night blue",
            "warm window glow"
          ],
          "style_opener": "A soft PAPER-CRAFT DIORAMA tableau at night. Layered cut-paper construction-paper shapes with visible scissor-cut edges and crisp paper shadows, lit by cool soft moonlight rather than saturated daylight. Pastel dreamscape palette (lavender night, mint, dusty pink, cream, soft sky, moonlight silver, warm window glow). Tiny paper stars, paper crescent moon, paper clouds. Cozy miniature-world quality \u2014 like a child's storybook diorama lit by a single desk lamp. When the protagonist appears, she is a young woman paper-doll figure with long wavy dusty-blonde paper hair, soft 3D modeling on hair + head (NOT flat 2D silhouette). MINIMAL FACE DISCIPLINE: lidded/closed soft paper eyes, NO mouth, NO nose, NO eyebrows \u2014 face is just hair-frame + a hint of eye-curve.",
          "negative_tail": "NO photoreal humans, NO photorealism, NO 3D render of the world itself (figure may have soft 3D hair only), NO AI-illustration look, NO smooth digital painting, NO oil-painting look, NO saturated daytime colors, NO neon, NO harsh shadows, NO sun, NO daylight, NO mouth on figure, NO nose on figure, NO text or words in the image. Vertical 9:16 composition."
        },
        "keyframe_engine": {
          "provider": "higgsfield",
          "model": "gpt_image_2",
          "resolution": "2k",
          "aspect_ratio": "9:16",
          "batch_size": 3
        },
        "clip_engine": {
          "provider": "higgsfield",
          "model": "kling3_0",
          "duration_sec": 5,
          "batch_size": 3,
          "motion_opener": "Subtle paper-craft diorama motion. Tiny micro-drift, soft breath, very slow subtle camera push or pulse. Paper textures hold. NO smooth AI-camera glide, NO photoreal CGI swoop, NO sudden movement, NO morphing. "
        },
        "captions": {
          "_comment": "Built from the song's OWN word timings (song.outputs.word_timings), NOT Whisper. Chunk ~3 words at natural lyric-phrase boundaries; accent words get the warm-glow color. The final outro 'with sleep again' is intentionally NOT captioned \u2014 the end-card overlay carries the tagline instead.",
          "source": "audio/words.json",
          "style": "lower_third_serif_italic",
          "font": "Georgia",
          "font_size": 56,
          "base_color": "#F5EDE0",
          "accent_color": "#FFD89C",
          "accent_words": [
            "fall",
            "love",
            "sleep",
            "again",
            "tap",
            "soft",
            "color",
            "worlds",
            "dreaming",
            "loo-nah"
          ],
          "chunk_words": 3
        },
        "tableaux": [
          {
            "id": "T01",
            "role": "INTRO",
            "t_start": 0.0,
            "t_end": 3.179,
            "lyric_anchor": "(intro / instrumental)",
            "caption": "",
            "keyframe_prompt": "Paper-craft bedroom diorama at night, viewed from above the bed. A paper-doll figure under a layered cut-paper duvet, paper-cut hair fanned on a paper pillow. A tiny paper rectangle (phone) rests on the figure's chest, glowing soft blue and casting cool light up onto the paper face. A small paper-cut window in the wall reveals a paper crescent moon. Pastel lavender-night walls. The phone glow is the hero.",
            "motion_hint": "The blue phone glow pulses ever so slightly, as if a feed is scrolling on the unseen screen. The paper moon hangs perfectly still."
          },
          {
            "id": "T02",
            "role": "PERSONAL_BEAT_1",
            "t_start": 3.179,
            "t_end": 6.299,
            "lyric_anchor": "two AM, and the screen still glows",
            "caption": "two AM and the / screen still glows",
            "keyframe_prompt": "Close-up on the paper-doll's paper face, lit blue from below by the paper phone. Paper eyes wide open, paper eyebrow furrowed in soft anxiety. A tiny ink-doodle sleep-Z hovers above the head, CROSSED OUT with a hand-drawn ink line. Dusty-pink paper pillow beneath.",
            "motion_hint": "The blue light flickers as the unseen feed scrolls. The crossed-out Z bobs gently."
          },
          {
            "id": "T03",
            "role": "PERSONAL_BEAT_2",
            "t_start": 6.299,
            "t_end": 8.319,
            "lyric_anchor": "endless feeds, and the night just slows",
            "caption": "endless feeds and the",
            "keyframe_prompt": "Macro overhead on the paper phone screen \u2014 paper rectangles representing feed posts scroll past in a blur, each a different pastel paper color. A paper thumb cuts in from the bottom right edge of frame. Cool moonlight silver fills the negative space around the phone.",
            "motion_hint": "The paper feed rectangles scroll upward in a continuous loop, blurring slightly. The paper thumb is still."
          },
          {
            "id": "T04",
            "role": "WIDE_ENSEMBLE",
            "t_start": 8.319,
            "t_end": 10.34,
            "lyric_anchor": "(verse close)",
            "caption": "night just slows",
            "keyframe_prompt": "Wide pull-back \u2014 the paper-craft bedroom sits inside a much larger dark paper-craft room, a diorama-within-a-diorama. Cool moonlight pools across the floor. A single warm-glow paper lamp at the bedside is the only warm color. Paper city skyline visible through a far window, all in flat cut paper. Sense of scale: tiny figure, vast quiet night.",
            "motion_hint": "Very slow continuous pull-back. The warm lamp flickers once. The city windows twinkle faintly."
          },
          {
            "id": "T05",
            "role": "KINETIC_CLOSE_1",
            "t_start": 10.34,
            "t_end": 12.639,
            "lyric_anchor": "then i tap, into something soft",
            "caption": "then i tap / into something soft",
            "keyframe_prompt": "Tight close-up of the paper thumb pressing down on a paper Lo\u00f3na app icon (a tiny cream-colored crescent moon on a deep night-blue square) on the paper phone screen. A small ripple of paper sparkle radiates outward from the tap point.",
            "motion_hint": "The thumb presses, the icon flexes inward then releases. Sparkle ripple expands outward."
          },
          {
            "id": "T06",
            "role": "KINETIC_CLOSE_2",
            "t_start": 12.639,
            "t_end": 14.059,
            "lyric_anchor": "all the noise of the day",
            "caption": "all the noise of",
            "keyframe_prompt": "Macro on the paper phone screen \u2014 a tiny paper-craft cottage diorama is now visible WITHIN the screen frame: paper roof, paper door, a paper-cut tree beside it, a paper path leading away. The cottage is in mint and dusty pink. Background of the bedroom recedes into blur.",
            "motion_hint": "The little paper cottage door opens a crack, revealing warm glow inside. A paper bird-shape flies past."
          },
          {
            "id": "T07",
            "role": "SUSPENDED_TIME",
            "t_start": 14.059,
            "t_end": 15.239,
            "lyric_anchor": "falls off",
            "caption": "the day falls off",
            "keyframe_prompt": "Slow-motion abstract beat \u2014 pastel paper clouds in dusty pink and lavender unfurl upward above an unseen scene. Paper-cut stars twinkle into existence one at a time. Tiny hand-drawn ink doodle sparkles burst at intervals. A frozen-time quality. Pure pattern, no character.",
            "motion_hint": "Clouds drift slowly upward. Stars pop in. Sparkles burst and fade. Frozen-time. (Match-cut into the hero reveal.)"
          },
          {
            "id": "T08",
            "role": "HOOK_HERO",
            "t_start": 15.239,
            "t_end": 18.579,
            "lyric_anchor": "fall in love with sleep again",
            "caption": "fall in love",
            "is_hook": true,
            "keyframe_prompt": "HERO FRAME. The bedroom walls have fallen away like paper opening up \u2014 the paper-doll figure now sits in a moonlit paper village. Small paper cottages with warm-glow paper windows line a paper path. A large paper crescent moon hangs overhead in a deep night-blue paper sky scattered with paper stars. Paper hills roll behind. The figure is small in frame, the WORLD is the hero. Cinematic dolly-out composition.",
            "motion_hint": "Subtle dolly-out reveals more of the village. The moon glows steadily. Cottage windows twinkle in sequence to the beat."
          },
          {
            "id": "T09",
            "role": "KINETIC_PEAK",
            "t_start": 18.579,
            "t_end": 20.459,
            "lyric_anchor": "color worlds till the dreaming ends",
            "caption": "with sleep again / color worlds till",
            "keyframe_prompt": "Slow aerial pan over the paper village from a higher angle. Every paper cottage window now glows warm. Paper smoke curls from paper chimneys. A tiny paper koi pond reflects the paper moon in soft silver. Paper trees with mint paper leaves. The whole village is alive in pastel light.",
            "motion_hint": "Slow continuous aerial pan forward, low altitude. Smoke curls rise. Koi pond ripples once. Windows pulse warmly."
          },
          {
            "id": "T10",
            "role": "PRODUCT_HERO",
            "t_start": 20.459,
            "t_end": 22.34,
            "lyric_anchor": "lo\u00f3na, lo\u00f3na",
            "caption": "the dreaming ends",
            "keyframe_prompt": "PRODUCT HERO \u2014 the paper phone alone, centered, floating in soft moonlight against a deep night-blue paper background. On the phone screen, the paper-craft sleepscape from T08-T09 is rendered in tiny precise detail. A paper Lo\u00f3na wordmark hovers in soft cream above the phone. Single key-light from above.",
            "motion_hint": "The phone rotates a few degrees on its axis. The sleepscape on screen glows softly. Moonlight shifts angle gently."
          },
          {
            "id": "T11",
            "role": "MICRO_MONTAGE",
            "t_start": 22.34,
            "t_end": 23.279,
            "lyric_anchor": "take me in",
            "caption": "lo\u00f3na, lo\u00f3na",
            "keyframe_prompt": "4-quadrant paper-craft contact sheet, each panel a different paper-craft sleepscape: TOP-LEFT a snowy paper cabin with paper smoke; TOP-RIGHT a paper lighthouse on a paper cliff at night; BOTTOM-LEFT a paper koi pond under a paper bridge; BOTTOM-RIGHT a paper treehouse in a paper canopy. Each panel a different pastel palette tone.",
            "motion_hint": "Each panel pulses warm-window glow in alternation to the music-box arpeggio. Subtle motion inside each."
          },
          {
            "id": "T12",
            "role": "ORIGIN_WINK",
            "t_start": 23.279,
            "t_end": 24.219,
            "lyric_anchor": "(outro line 1)",
            "caption": "take me in",
            "keyframe_prompt": "A paper-craft moon-and-stars mobile spins slowly above an empty paper crib in a soft pastel nursery. The mobile is in cream and warm window glow against a lavender wall. Paper sheep on the crib bedding. A nostalgic callback \u2014 Spanish 'luna' = moon, the brand's name origin. Slightly heavier paper grain.",
            "motion_hint": "The mobile rotates very slowly. A paper sheep stirs."
          },
          {
            "id": "T13",
            "role": "TWILIGHT_OUTRO",
            "t_start": 24.219,
            "t_end": 26.239,
            "lyric_anchor": "fall in love with sleep",
            "caption": "fall in love",
            "keyframe_prompt": "Return to the original paper-craft bedroom from T01 \u2014 the paper-doll figure is now asleep, paper duvet rising and falling in a slow breath rhythm. The paper phone is face-down on the paper nightstand. The window reveals the paper crescent moon, now bigger and softer. A single warm lamp glow throws long paper shadows. Reflective, peaceful.",
            "motion_hint": "The paper duvet rises and falls in two slow breath cycles. The moon hangs perfectly still."
          },
          {
            "id": "T14",
            "role": "END_CARD",
            "t_start": 26.239,
            "t_end": 28.0,
            "lyric_anchor": "again",
            "caption": "",
            "keyframe_prompt": "END CARD BASE \u2014 deep night-blue paper background with a soft paper crescent moon and three small paper stars in a moonlight spotlight. Brand lockup composited on top via PIL (NOT AI-rendered).",
            "motion_hint": "Static hold. The paper moon sways imperceptibly on a hidden paper thread."
          }
        ],
        "end_card": {
          "_comment": "Brand lockup composited via PIL from the REAL brand asset (the Lo\u00f3na app icon pulled from iTunes Search API) \u2014 NEVER AI-rendered text (LEARNINGS L4). Overlaid on the final tableau window.",
          "engine": "pil",
          "script": "build_end_card.py",
          "brand_asset": "brand-assets/app-icon-1024.jpg",
          "wordmark_text": "Lo\u00f3na",
          "wordmark_font": "Georgia Bold",
          "tagline_text": "fall in love with sleep again",
          "tagline_font": "Georgia Italic",
          "cta_text": "Download on the App Store",
          "background": "deep brand purple (#2A1F52) gradient + soft stars + circular app icon with warm-glow halo",
          "overlay_window_sec": [
            26.24,
            28.0
          ]
        },
        "audio_mix": {
          "loudness_lufs": -14,
          "climax_beat_id": "T08",
          "climax_boost_db": 2.0,
          "duck_under_vo": false
        },
        "post_production": {
          "music": {
            "default": "on",
            "note": "the sung generated ElevenLabs track IS the bed (force_instrumental false)"
          },
          "captions": {
            "default": "on",
            "note": "lyric-synced captions from the song's word timings, default on"
          },
          "end_card": {
            "default": "on",
            "note": "PIL end card from the real app icon, default on"
          }
        }
      }
      
    • PIPELINE.md 4.8 KB
      # Pipeline — song-driven music-video
      
      How `config.example.json` maps to the real production steps. This molecule ships a
      **config + this map**, not a bundled runner: the worked example (Loóna "Fall In Love With
      Sleep Again") was produced by the control-plane orchestrator, whose per-step scripts live
      in `clients/loona/ad-runs/run-01-run1/working/`. Reference those scripts directly, or drive
      the whole run via `video-orchestrator-with-control-plane`.
      
      The five steps run **in order** because each depends on the last: the song sets the
      timeline, the timeline + look pack drive the keyframes, the keyframes seed the clips, the
      song's word timings drive the captions, and assembly stitches all of it.
      
      ## 1. Song → ElevenLabs `music_v1`  (config: `song`)  [PAID]
      
      **Generate the song FIRST — it sets the timeline.** Feed `song.structure` (the
      intro/verse/pre-chorus/chorus/outro sections, each with `duration_ms` + exact `lines`) and
      `song.prompt` (the vibe) to ElevenLabs `music_v1`. The call returns:
      - `audio/music.mp3` — the sung track (the narration; **no separate VO**).
      - `audio/music_metadata.json` — the `composition_plan` echo **plus `words_timestamps`**
        (word-level start/end in ms).
      - `audio/words.json` — the word timings normalized to seconds (used by captions).
      
      Then derive `working/timeline.json` — `[tableauId, tStart, tEnd]` per beat — by snapping
      each tableau boundary to the lyric-phrase edges in the returned word timings. The hook
      word (`song.hook_word` = "fall") lands at ~15.24s, the chorus drop; the HOOK_HERO tableau
      (T08) is timed to it. No artist names in `song.prompt` (ElevenLabs ToS filter).
      
      ## 2. Keyframes → Higgsfield `gpt_image_2`  (config: `tableaux[].keyframe_prompt`, `look_pack`, `keyframe_engine`)  [PAID]
      
      `working/render_keyframes.py` reads the scene list and, per tableau, builds the prompt as
      `look_pack.style_opener + tableaux[i].keyframe_prompt + look_pack.negative_tail`, then calls
      `higgsfield generate create gpt_image_2 --aspect_ratio 9:16 --resolution 2k --wait --json`.
      Parallel batches of 3 (Higgsfield burst-credit reserve). One PNG per beat → `keyframes/<id>/v1.png`
      (2k 9:16 ≈ 1520×2688). Idempotent — skips beats that already have a keyframe. The single
      `look_pack` is what makes N beats read as one film; review all N before step 3.
      
      ## 3. Clips → Higgsfield `kling3_0` i2v  (config: `tableaux[].motion_hint`, `clip_engine`)  [PAID]
      
      `working/render_clips.py` reads each keyframe + its `motion_hint`, builds the prompt as
      `clip_engine.motion_opener + motion_hint`, and calls
      `higgsfield generate create kling3_0 --start-image keyframes/<id>/v1.png --duration 5 --wait --json`.
      One ~5s clip per beat → `clips/<id>/v1.mp4`. Batches of 3; stops cleanly on
      `not_enough_credits`. Kling 3.0 (not Seedance) holds the paper-craft register and the
      character-face discipline through the motion.
      
      ## 4. Captions → `build_captions_v2.py`  (config: `captions`)
      
      `working/build_captions_v2.py` reads `audio/words.json` (the song's OWN word timings — **not
      Whisper**; Whisper on sung audio returns "🎵 Music Playing 🎵"). It hand-chunks ~3 words at
      lyric-phrase boundaries, times each caption event to its first/last word, and colors
      `captions.accent_words` in the warm-glow `accent_color` (#FFD89C) against the cream
      `base_color` (#F5EDE0), lower-third Georgia italic → `finals/master-final-v2.ass`. The final
      outro "with sleep again" is intentionally left uncaptioned so it doesn't double-stack with
      the end-card tagline overlay.
      
      ## 5. End card + assembly → `build_end_card.py` + `promote_master*.py`  (config: `end_card`, `audio_mix`)
      
      - `working/build_end_card.py` composites the brand lockup via **PIL** from the REAL brand
        asset (`brand-assets/app-icon-1024.jpg`, pulled from the iTunes Search API): brand-purple
        gradient + soft stars + circular app icon with a warm-glow halo + "Loóna" wordmark
        (Georgia Bold) + tagline (Georgia Italic) + App Store CTA → `working/end-card.png`.
        **Never AI-render brand text** (LEARNINGS L4 — a prior end card rendered "therapits").
      - Assembly (`working/promote_master.py` → `_v2` → `_v3`, the version chain): cut each clip
        to its lyric window from `timeline.json`, hard-concat on the beat (one match-cut into T08),
        burn the caption ASS via libass, overlay the PIL end card on the final window
        (`end_card.overlay_window_sec` = 26.24–28.0), mux `audio/music.mp3`, boost the climax beat
        (`audio_mix.climax_boost_db` on T08), and loudnorm to `audio_mix.loudness_lufs` (−14 LUFS).
        Output → `renders/master-v3.mp4` (1080×1920, 30fps, h264+aac, 28s). The `promote_master*.py`
        scripts also register the render into `production/asset-manifest.json`,
        `production/render-outputs.json`, and `history/versions.json` for the control-plane app.
      
      Re-cuts (new caption chunking, swapped end card, re-timed windows) reuse the existing
      keyframes/clips/song and cost **$0** — only steps 1–3 spend.
      
    • README.md 3.1 KB
      # render-song-mv scripts — the FREE assembly
      
      `render-song-mv` is the **deterministic, $0 assembly stage** of the song-driven
      music-video format. The three paid stages (the sung song, the N keyframes, the N i2v
      clips) are separate capabilities — `create-music-elevenlabs`, `create-image-fal`,
      `create-video-fal`. This capability spends nothing: it takes the delivered song +
      `words.json` + one clip per tableau and stitches the finished master. Re-cuts (new
      caption chunking, a swapped end card, re-timed windows) reuse the existing song / keyframes
      / clips and cost **$0** — only the paid capabilities spend.
      
      `config.example.json` is the worked example (Loóna "Fall In Love With Sleep Again", 28s
      paper-craft 9:16). `PIPELINE.md` maps every config block to its source step. This README
      documents the four FREE assembly pieces that `render-song-mv` owns.
      
      ## 1. Lyric-synced captions — from the song's OWN word timings (script-window, NOT Whisper)
      
      Captions come from `audio/words.json` — the word-level timestamps the music model returns
      **with** the song — never from Whisper. Whisper on a sung track returns "🎵 Music Playing 🎵",
      so it cannot time captions to lyrics; the script-window approach reads the song's own words
      instead. The builder hand-chunks ~3 words at natural lyric-phrase boundaries (`captions.chunk_words`),
      times each caption event to its first/last word, and colors the `captions.accent_words` in
      the warm-glow `accent_color` against the `base_color` (lower-third serif italic) → an ASS
      subtitle file. The hero/hook chunk is timed so the payoff word (`song.hook_word`) lands on
      the chorus drop; the outro line is intentionally left uncaptioned so it doesn't double-stack
      with the end-card tagline.
      
      ## 2. Per-scene clip assembly to the song timeline
      
      The song sets the timeline. Each tableau boundary in `timeline.json` was snapped to a
      lyric-phrase edge in the returned word timings, so assembly cuts each per-tableau clip to
      its lyric window and hard-concats the clips **on the beat** (one optional match-cut into the
      hook-hero reveal). No dissolves. The N clips + their windows are a pure function of the song,
      so the whole cut is deterministic and repeatable.
      
      ## 3. PIL end card — from the real app icon, no AI text
      
      The brand lockup is composited with **PIL** from the brand's REAL asset (e.g. the app icon
      pulled from the iTunes Search API): brand-color gradient + soft stars + the circular app
      icon with a warm-glow halo + wordmark + tagline + CTA → a PNG overlaid on the final window
      (`end_card.overlay_window_sec`). The brand text is **never** AI-rendered — a diffusion model
      garbles a wordmark ("therapits"), so the lockup is drawn deterministically from the real
      asset every time.
      
      ## 4. FFmpeg composite
      
      FFmpeg stitches the whole master: cut each clip to its lyric window, hard-concat on the beat,
      burn the caption ASS via libass, overlay the PIL end-card PNG on the final window, mux
      `audio/music.mp3` (the sung track IS the bed — no separate VO), boost the climax beat
      (`audio_mix.climax_boost_db`), and loudnorm to `audio_mix.loudness_lufs` (−14 LUFS). Output
      is a 1080×1920 h264+aac master. Deterministic, no paid calls, no keys.
      
  • tests
    • smoke-test.md 1.2 KB
      # Smoke Test
      
      Given a delivered sung song (mp3 + `audio/words.json`), one keyframe + one i2v clip per
      tableau, and the brand's real app icon, `render-song-mv` assembles the master: cut each clip
      to its lyric window, hard-concat on the beat, build lyric-synced captions, composite the PIL
      end card, mux the song → 1080×1920 h264+aac (~28s).
      
      Pass when the assembly runs to a valid MP4 and:
      - captions are built from the song's OWN `words.json` (script-window, NOT Whisper) and every
        chunk tracks the sung word;
      - exactly ONE hook tableau lands the payoff word (`song.hook_word`) on the chorus drop, with
        that word accent-colored;
      - clips are cut to their lyric windows and hard-cut on the beat (no dissolves bar the one hero
        match-cut); the look pack holds across all N beats;
      - the end card is composited via PIL from the real app icon (no AI-rendered brand text);
      - the sung track carries with no separate VO, loudnormed to −14 LUFS;
      - **no paid call is made** — the song / keyframes / clips come from the paid capabilities
        (create-music-elevenlabs / create-image-fal / create-video-fal); this assembly is $0 and a
        re-cut reuses the existing assets. For the paid caps the call is proxy-routed (bills the
        agent, no direct provider host).
      
  • SKILL.md 3.8 KB
    ---
    name: render-song-mv
    description: Assemble a song-driven music-video ad from a config — a generated sung track carries the whole narration across N tableaux (one keyframe -> one i2v clip per lyric beat) with NO separate voiceover, captions synced to the song's OWN word timings (script-window, never Whisper) and the hook word landing on the chorus drop, closed on a PIL brand end card. This is the FREE deterministic assembly stage (clip cut-to-timeline + captions + end card + FFmpeg composite); the song, keyframes, and clips come from create-music-elevenlabs / create-image-fal / create-video-fal. Use for the song-driven-music-video format.
    status: active
    ---
    
    # render-song-mv
    
    Assemble a song-driven music-video ad from a config: a purpose-written, **sung** song is
    the entire narration (no separate voiceover), and every visual beat is timed to the lyrics.
    The delivered song sets the timeline; N tableaux (one keyframe → one image-to-video clip per
    lyric beat, all in a single look pack) are cut to their lyric windows and hard-concatenated
    on the beat, captions are built from the song's OWN word timings with the hook line landing
    on the chorus drop, and the spot closes on a PIL brand end card. It reads like a tiny animated
    music video, not a demo. `scripts/config.example.json` is the worked example (Loóna "Fall In
    Love With Sleep Again", 28s paper-craft 9:16); `scripts/PIPELINE.md` maps every config block
    to its step and `scripts/README.md` documents the free assembly.
    
    ## Run
    
    This is the **FREE, deterministic** assembly stage — it spends nothing. The three paid
    inputs are separate capabilities: the sung song (`create-music-elevenlabs`, music_v1,
    force_instrumental FALSE — the lyrics ARE the script, returns mp3 + `words.json`), one
    keyframe per tableau (`create-image-fal`), and one Kling 3.0 i2v clip per tableau
    (`create-video-fal`). Given the delivered song + `words.json` + one clip per beat,
    `render-song-mv` cuts each clip to its lyric window, hard-concats on the beat, builds the
    lyric-synced captions, composites the PIL end card, and muxes → the master. Re-cuts reuse
    the existing song / keyframes / clips and cost **$0**.
    
    ## Contract (the free assembly)
    
    - **The sung song carries the narration — no separate VO.** The generated ElevenLabs track
      IS the bed and the script (`force_instrumental` false); do not add a spoken voiceover or a
      second music bed.
    - **Plan the timeline AROUND the delivered song.** The song is generated first and reshapes/
      overshoots length; snap every tableau boundary to the lyric-phrase edges in the returned
      word timings (`timeline.json`) — never trim the song to a pre-planned grid.
    - **Captions from the song's OWN word timings, not Whisper (script-window).** Chunk
      `audio/words.json` (~3 words at lyric boundaries); accent words get the warm-glow color.
      Whisper on sung audio returns "🎵 Music Playing 🎵", so it can't caption lyrics.
    - **Land the hook on the chorus drop.** Exactly ONE hero tableau (`is_hook`) is timed so the
      payoff word (`song.hook_word`) sits on the chorus drop; accent that word in the captions.
    - **One look pack for consistency.** A single `style_opener` + `negative_tail` + palette drives
      every keyframe so N beats read as one film; no morph within a clip.
    - **Hard cuts on the beat.** Cut each clip to its lyric window and hard-concat — no dissolves
      (one optional match-cut into the hero reveal).
    - **PIL end card from the real app icon — never AI-render brand text.** The lockup is
      composited deterministically (brand gradient + circular app icon + wordmark + tagline + CTA)
      from the brand's real asset; a diffusion model garbles a wordmark.
    - **FFmpeg composite, deterministic, FREE.** Burn the caption ASS, overlay the end-card PNG on
      the final window, mux the song, boost the climax beat, loudnorm to −14 LUFS → 1080×1920
      h264+aac. No paid calls, no keys.
    
  • skill.meta.json 305 B
    {
      "slug": "render-song-mv",
      "category": "capabilities",
      "domain": "ads",
      "tags": [
        "ads"
      ],
      "installation": {
        "base_command": "npx goose-skills install render-song-mv",
        "supports": [
          "claude",
          "cursor",
          "codex"
        ]
      },
      "requires_skills": [
        "watch"
      ]
    }
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related