ChatGPT Claude Codex CLI Cohere Cursor DeepSeek Gemini GitHub Copilot GLM Grok Kimi Llama MiniMax Mistral OpenAI opencode Skill

gemini-omni-flash-api

Use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,..

LLM Mart · 0 points · 29 views 2 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download sickn33-agentic-awesome-skills-skills_gemini-omni-flash-api-286166a.zip · 20 KB
Part of sickn33/agentic-awesome-skills — 427 skills
This skill couldn't be refreshed from GitHub on the last check — you're seeing the last imported snapshot.

Install

skills CLI npx skills add https://github.com/sickn33/agentic-awesome-skills/tree/main/skills/gemini-omni-flash-api
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install sickn33-agentic-awesome-skills@llmmart
Git git clone https://github.com/sickn33/agentic-awesome-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole sickn33/agentic-awesome-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Gemini Omni Flash Skill

When to Use

Use this skill when you need use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,...

This skill uses the Gemini Omni Flash model (gemini-omni-flash-preview) to perform text to video generation, image to video generation and video editing.

Warning

Important Regional Restrictions: Uploading videos to use for video edits is NOT available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (total_output_tokens: 0 or no video content), it is likely due to this restriction.

Core capabilities

  1. Video editing and refinement: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting.
  2. Text to video: Generating videos from a text prompt.
  3. First-frame to video: Generating videos from a single input image.
  4. Image-referenced generation: Using style, character, or object references from images to guide video generation.

Workflow

  1. Analyze request: Determine the target task (e.g., first-frame-to-video, reference-guided editing) and identify any input media assets.

  2. Run SDK scripts:

    • Directly run the appropriate utility (scripts/video/generate_video.py or scripts/upload_file.py).
    • Configure settings like --aspect-ratio (e.g. 16:9, 9:16) and --duration (any integer between 3 and 10 seconds, e.g. 3, 5, 10).
  3. Retrieve and process output: Outputs are saved to the local filesystem (e.g. media/). Report back the completed media path to the user.

Reference Documentation

  • Interactions API: All operations and state management for the Gemini Omni Flash model (gemini-omni-flash-preview) are handled via the Interactions API.
  • Files API: Input media files (such as reference images and videos) must be uploaded via the Files API first before being referenced in generations. The uploaded file URI and MIME type are then included in the interactions.create input parts array.
  • Interactions API Skill Reference: Platform-wide guidelines, current model specifications, and SDK usage rules for the Interactions API.

Dependencies and Prerequisites

  • Python SDK (google-genai): Requires google-genai >= 2.10.0 (Python) to support the new interactions client attribute. Install or upgrade using:
    pip install -U google-genai
    
  • Python Runtime: Requires Python >= 3.10 (for compatibility with modern google-genai SDK types and methods).
  • ffmpeg & ffprobe: prep_video.py, inspect_video.py, and generate_video.py (when stripping audio via --strip-audio) require ffmpeg and ffprobe binaries installed and available in your system PATH.

Available scripts

Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API.

  1. upload_file.py: Uploads local media (images and videos) to the Files API and polls until ACTIVE. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing with prep_video.py first to speed up the upload.

    ./scripts/upload_file.py path/to/image.png
    
  2. generate_video.py: Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload.

    • Text to video:

      ./scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4
      
    • Image to video (first frame and reference):

      ./scripts/video/generate_video.py "The waves crash against the shore." --image reference.png --output media/waves.mp4
      
    • Video interpolation:

      Provide exactly two images as keyframes to generate a transition video between them:

      ./scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --image start.png --image end.png --output media/interpolation.mp4
      
    • Video editing (keep original audio):

      ./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --output media/anime_style.mp4
      
    • Video editing (regenerate all audio from scratch):

      ./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --strip-audio --output media/anime_style_new_audio.mp4
      
    • Turn-by-turn video editing (edit previous interaction):

      Edit a prior video generation without re-uploading assets by passing the interaction ID:

      ./scripts/video/generate_video.py "Change the setting to a snowy winter wonderland." --previous-interaction-id "abc123xyz..." --output media/winter_wonderland.mp4
      
    • Parallel batch execution (prompts file): Run multiple prompts from a line-by-line text file concurrently:

      ./scripts/video/generate_video.py --prompts-file prompts.txt --concurrency 3
      
    • Parallel batch execution (JSON config): Execute fully configured, distinct generation and editing jobs in parallel:

      ./scripts/video/generate_video.py --batch jobs.json --concurrency 3
      

      Example jobs.json:

      [
        {
          "prompt": "Transform the style to Japanese anime.",
          "video": "input.mp4",
          "output": "media/anime_style.mp4",
          "strip_audio": false,
          "aspect_ratio": "16:9"
        },
        {
          "prompt": "A smooth timelapse from sunrise to sunset.",
          "image": ["start.png", "end.png"],
          "output": "media/interpolation.mp4"
        }
      ]
      
  3. inspect_video.py: Inspects a local video file (using ffprobe) to check its duration, resolution, frame rate (FPS), audio stream presence, and format details.

    ./scripts/video/inspect_video.py media/output.mp4
    
    • To get a pre-parsed, structured JSON summary:

      ./scripts/video/inspect_video.py media/output.mp4 --json
      
    • To get the complete, unmodified ffprobe raw JSON dump:

      ./scripts/video/inspect_video.py media/output.mp4 --raw
      
  4. prep_video.py: Normalizes, trims, and formats any video file to fit standard Gemini Omni Flash generation and editing limits. It handles timecode-based trimming, optional frame rate conversion, and proportional scaling of large videos (max 1280x720 for landscape, 720x1280 for portrait) to optimize upload times without stretching. If the video is longer than 10 seconds and the script is run interactively (in a TTY), it prompts the user to select the first 10s, last 10s, or enter a custom timecode (defaulting to the first 10s).

    • Trim first 10s (default):
     ./scripts/video/prep_video.py path/to/source.mp4
    

    or explicitly specify the start and duration:

    ./scripts/video/prep_video.py path/to/source.mp4 --start 0 --duration 10
    
    • Trim last 10s (automatically calculates starting point based on source length):

      ./scripts/video/prep_video.py path/to/source.mp4 --start last
      
    • Trim 10s starting at specific timecode (MM:SS or HH:MM:SS):

      ./scripts/video/prep_video.py path/to/source.mp4 --start 00:03 --output media/custom.mp4
      
    • Custom frame rate and resolution:

      ./scripts/video/prep_video.py path/to/source.mp4 --fps 30 --resolution 1920x1080
      
    • Strip audio for audio regeneration:

      ./scripts/video/prep_video.py path/to/source.mp4 --strip-audio --output media/video_with_no_audio.mp4
      

Using tags in prompts to set image roles

You can use tags in your prompt to make it clear whether each uploaded media is an initial frame or a reference.

1. Simple tags (recommended)

For simple cases where image roles are clear from the prompt, you can bind images to roles directly:

  • <FIRST_FRAME>: Use the image as the starting frame of the video, for example: <FIRST_FRAME> a woman is walking
  • <IMAGE_REF_N>: Use the image as a reference, for example: in the style of <IMAGE_REF_0> a woman <IMAGE_REF_1> is walking (combines style reference from the first image and subject reference from the second image). Image references start from 0.

An example with 6 reference images:

[0-3s] A studio fashion sequence. Starting with woman <IMAGE_REF_0>, she is holding <IMAGE_REF_1>
[3-6s] Then we see the man <IMAGE_REF_2> holding <IMAGE_REF_3>
[6-10s] And finally another woman <IMAGE_REF_4> who is holding <IMAGE_REF_5> while walking.

2. Explicitly declare sources and references

For more complex cases with multiple images and multiple roles, you can use explicit prefix tags paired with natural language instruction suffixes.

  • Declaring sources and reference images:

    • [# Sources <FIRST_FRAME>@Image1] will use the first image as the starting frame.
    • [# References <IMAGE_REF_0>@Image1] will use the first image as a reference.
    • [# References <IMAGE_REF_1>@Image2] will use the second image as a reference.
    • [# References <IMAGE_REF_0>@Image1 <IMAGE_REF_1>@Image2] will use both images as references.
    • [# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2] will use the first image as the starting frame and the second image as a reference.
  • Guiding instructions: Add guiding instructions at the end of your prompt:

    • For starting frame: "Use the given image as the starting frame."
    • For reference images: "Use the given image(s) as references for video generation. The images should not be used as literal initial frames."
  • Example Expanded Prompt:

    [# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2] a woman <IMAGE_REF_0> is walking. Use Image1 as the starting frame. Use Image2 as a reference for the video generation.
    

Audio handling in video editing

When editing a source video that contains audio, you must choose between keeping the original audio or regenerating all audio from scratch.

  • Keep original audio: By default, Gemini Omni Flash preserves the existing audio layer (though it may modify or adapt it slightly during generation). Use this when the original background music, dialogue, or sound effects are desired.

  • Regenerate all audio from scratch: If you want Gemini Omni Flash to re-create a brand-new audio layer tailored to the new visual style or prompt, you must upload the video with its audio stream stripped out. If any audio stream is present, Gemini Omni Flash will attempt to preserve/modify it instead of starting from scratch.

    • Use --strip-audio (or -a) when pre-processing with scripts/video/prep_video.py or executing scripts/video/generate_video.py.
    • This forces Gemini Omni Flash to perform full audio generation.

Prompting Gemini Omni Flash

Single scene

By default Gemini Omni Flash will try to create a video with a few different shots. It'll attempt to craft an interesting narrative based on the prompt.

If you need the output video to contain a single scene, you must prompt for that:

  • In a single unbroken scene
  • In a single continuous shot
  • No scene cuts

For example:

Continuous, unbroken handheld shot of a fluffy tabby cat sitting on a sunny windowsill, looking out into a leafy garden. The cat's tail twitches slowly, and its ears rotate slightly toward ambient noises. Sunbeams illuminate dust motes in the air. Sound design: Gentle breeze, distant bird chirps, quiet mechanical purring. No dialogue.

Removing unwanted elements

If generations contain things you don't want, you can include simple negatives to avoid them:

  • No dialogue
  • No embellishments
  • No extra sound effects

Prompts for editing

Simple prompts work best for editing. Overly descriptive prompts can lead to unintended changes.

For example:

  • Make this video anime
  • Make the phone invisible
  • Put a fashionable hat on this person
  • Change the lighting to be more dramatic
  • Change the text on the sign to say "Gemini Omni Flash"
  • Add a cat that jumps onto his lap, he begins to pet it

When editing a specific aspect of the video, it can help to include: "Keep everything else the same".

Prompting the audio

By default the model will try to generate an appropriate audio track for a video. This might not always be what you want. You can use your prompt to describe the type of audio you want. This is especially important if you want music in your video:

  • Include calm background music
  • The video has a high energy techno beat
  • The audio is a low tinny radio broadcast in the background, playing a song
  • Audio design: [a description of the audio you want]

When things should happen

You can prompt for things to happen at specific times in the video, there is no precise syntax needed and you can use natural language. This is especially useful in creating your own scene cuts, rhythm or rapid fire sequences.

Simple examples:

  • after 3 seconds, a woman enters the scene
  • at 5s the chorus starts in the background audio
  • every 2s cut to a new frame
  • in a rapid fire sequence, every half a second (12 frames at 24fps) change the scene to a new location

You can also use a timecode syntax:

[0-3s] A person is walking
[3-6s] They stop and turn around
[6-10s] They start running

Meta prompting

Rather than specifying everything directly in a prompt, you can ask the model to pay attention to certain things. You can give Gemini Omni Flash these sorts of prompts verbatim:

  • Consider micro-detail, expression and timing to create a very rich, detailed but entirely natural scene.
  • Be extremely detailed in your descriptions of characters and environments. Apply costume design principles to characters. Be very specific about the people, items and objects in the scene.
  • Include plenty of appropriate detail in the background elements to make the scene feel realistic and natural.
  • Make a rapid fire video that shows a different rare [thing] every 1s, upbeat music, include text to label the thing.

Text in videos works really well

Unlike previous video models, text in Gemini Omni Flash videos works really well. You can include decent amounts of text in your video and it will be rendered in a way that is correct and readable. If there will be naturally occurring text in your video, even in background elements, it can help to define what it should say.

For example:

  • One word on the screen at a time: "did, you, know, that, Omni, can, do, awesome, text?" Each word appears for 1s with a different animated style. No dialogue.
  • There is a street sign that says: "This is an AI generation by Omni", there is a storefront that says: "All you need AI", there's a car with the number plate: "OMN111"

Limitations

  • Use this skill only when the task clearly matches its upstream product or API scope.
  • Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
  • Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
Files (agentic-awesome-skills)
  • scripts
    • video
      • generate_video.py 19.5 KB
        #!/usr/bin/env python3
        """
        Generates and edits videos using the Gemini Omni Flash model via the google-genai Interactions API.
        Can automatically upload local media references using the Files API.
        Supports parallel execution of multiple generations using Python standard library.
        Uses the official google-genai SDK.
        """
        
        import argparse
        from concurrent.futures import ThreadPoolExecutor, as_completed
        import json
        import os
        import re
        import sys
        import time
        import urllib.request
        import urllib.error
        import urllib.parse
        import uuid
        from google import genai
        
        # Load local upload helper logic inline to prevent dependency issues
        sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
        from upload_file import upload_file, wait_for_active
        from pathlib import Path
        
        
        def safe_user_path(path_value, base_dir="."):
            """Resolve a CLI path under the current workspace."""
            if base_dir != ".":
                raise ValueError("Custom base directories are not supported for CLI paths")
            base_path = Path.cwd().resolve()
            resolved_path = Path(path_value).expanduser().resolve()
            try:
                resolved_path.relative_to(base_path)
            except ValueError as exc:
                raise ValueError(f"Path escapes allowed directory: {path_value}") from exc
            return resolved_path
        
        def get_api_key(args):
            """Retrieves API key from command args or environment."""
            if args.api_key:
                return args.api_key
            return os.environ.get("GEMINI_API_KEY")
        
        FILE_ID_RE = re.compile(r'^[A-Za-z0-9_-]+$')
        
        
        def extract_file_id(uri):
            """Returns a Gemini File API id from a local reference or trusted API URL."""
            if not uri:
                return None
            if uri.startswith("files/"):
                file_id = uri.removeprefix("files/")
                return file_id if FILE_ID_RE.fullmatch(file_id) else None
        
            parsed = urllib.parse.urlparse(uri)
            if parsed.scheme != "https" or parsed.netloc != "generativelanguage.googleapis.com":
                return None
            path_match = re.fullmatch(r'/files/([A-Za-z0-9_-]+)', parsed.path)
            return path_match.group(1) if path_match else None
        
        
        def is_file_uri(uri):
            """Returns True if the string is a standard Gemini File URI."""
            return extract_file_id(uri) is not None
        
        def normalize_file_uri(uri):
            """Normalizes any File API URI/reference to the standard https://generativelanguage.googleapis.com/files/{id} format."""
            file_id = extract_file_id(uri)
            if file_id:
                return f"https://generativelanguage.googleapis.com/files/{file_id}"
            return uri
        
        
        def media_download_url(file_uri):
            """Build a media URL only for validated Gemini File API references."""
            file_id = extract_file_id(file_uri)
            if not file_id:
                raise ValueError("Generated video URI must be a Gemini File API reference.")
            return f"https://generativelanguage.googleapis.com/files/{file_id}?alt=media"
        
        def slugify(text):
            """Converts a text prompt into a safe, descriptive filename slug."""
            text = text.lower()
            text = re.sub(r'[^a-z0-9]+', '_', text)
            return text.strip('_')[:50]
        
        def parse_and_validate_duration(value):
            """Parses and formats a duration integer between 3 and 10 with optional 's' suffix."""
            if value is None:
                return None
            if isinstance(value, (int, float)):
                val = float(value)
            else:
                clean_value = str(value).strip().lower()
                if clean_value in ('none', ''):
                    return None
                if clean_value.endswith('s'):
                    clean_value = clean_value[:-1]
                try:
                    val = float(clean_value)
                except ValueError:
                    raise ValueError(f"Invalid duration value: '{value}'. Must be an integer (e.g., 5, 10).")
        
            if not val.is_integer():
                raise ValueError(f"Duration must be an integer, not a float (e.g., got {value}).")
        
            val_int = int(val)
            if val_int < 3 or val_int > 10:
                raise ValueError(f"Duration must be between 3 (inclusive) and 10 (inclusive) seconds. Got {val_int}.")
        
            return f"{val_int}s"
        
        def argparse_duration_type(value):
            """argparse type converter for validating duration."""
            if value is None or str(value).strip().lower() in ('none', ''):
                return None
            try:
                return parse_and_validate_duration(value)
            except ValueError as e:
                raise argparse.ArgumentTypeError(str(e))
        
        def resolve_or_upload_asset(asset_path, mime_type, api_key, strip_audio=False):
            """
            If asset_path is a File API URI, returns it directly (normalized).
            If it is a local file path, uploads it and returns its File API URI (normalized).
            """
            if not asset_path:
                return None, None
        
            if is_file_uri(asset_path):
                normalized = normalize_file_uri(asset_path)
                print(f"Using existing File URI: {normalized}")
                if strip_audio:
                    print("Warning: --strip-audio was specified but the video input is an existing remote File URI. "
                          "Audio cannot be stripped from remote files automatically.")
                return normalized, mime_type
        
            if os.path.exists(asset_path):
                upload_path = asset_path
                temp_stripped_path = None
        
                if strip_audio:
                    print(f"Detected local asset path '{asset_path}'. Stripping audio before upload...")
        
                    # Check if ffmpeg is available
                    import subprocess
                    try:
                        subprocess.run(["ffmpeg", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)
                    except (subprocess.SubprocessError, FileNotFoundError):
                        raise RuntimeError(
                            "Error: ffmpeg is not installed or not found in system PATH. "
                            "ffmpeg is required to strip audio from local videos."
                        )
        
                    try:
                        os.makedirs("media", exist_ok=True)
                        base_name = os.path.basename(asset_path)
                        name, ext = os.path.splitext(base_name)
                        temp_stripped_path = os.path.join("media", f"temp_stripped_{name}_{uuid.uuid4().hex}{ext}")
        
                        # Fast stream-copy audio stripping
                        cmd = ["ffmpeg", "-y", "-i", asset_path, "-c:v", "copy", "-an", temp_stripped_path]
                        subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)
        
                        print(f"Successfully stripped audio. Temporary video file created at: {temp_stripped_path}")
                        upload_path = temp_stripped_path
                    except Exception as e:
                        print(f"Error stripping audio using ffmpeg: {e}", file=sys.stderr)
                        print("Falling back to uploading the original video with audio.", file=sys.stderr)
        
                print(f"Uploading asset '{upload_path}'...")
                file_meta = upload_file(upload_path, api_key=api_key)
                file_name = file_meta.get("name")
                # Wait for file to become active
                file_meta = wait_for_active(file_name, api_key)
                normalized = normalize_file_uri(file_meta.get("uri"))
        
                # Clean up temporary stripped file if we created one
                if temp_stripped_path and os.path.exists(temp_stripped_path):
                    try:
                        safe_user_path(temp_stripped_path).unlink()
                        print(f"Cleaned up temporary video file: {temp_stripped_path}")
                    except Exception as e:
                        print(f"Warning: Failed to remove temporary file {temp_stripped_path}: {e}", file=sys.stderr)
        
                # Handle both mimeType and mime_type key formats returned from upload_file
                returned_mime = file_meta.get("mimeType") or file_meta.get("mime_type")
                return normalized, returned_mime
            else:
                raise FileNotFoundError(f"Asset path '{asset_path}' is neither a valid File API URI nor a local file path.")
        
        def download_video_file(file_uri, output_path, api_key):
            """Downloads generated video file from URI using alt=media standard in a memory-safe, chunked manner."""
            download_url = media_download_url(file_uri)
        
            print(f"Downloading video from {file_uri} to {output_path} in chunked mode...")
            req = urllib.request.Request(download_url)
            req.add_header("x-goog-api-key", api_key)
        
            try:
                with urllib.request.urlopen(req, timeout=480) as resp:
                    parent_dir = os.path.dirname(output_path)
                    if parent_dir:
                        os.makedirs(parent_dir, exist_ok=True)
        
                    with safe_user_path(output_path).open("wb") as f:
                        while True:
                            chunk = resp.read(8192)
                            if not chunk:
                                break
                            f.write(chunk)
                print(f"Video successfully saved to: {output_path}")
            except urllib.error.HTTPError as e:
                raise RuntimeError(f"Error downloading video file: {e.code} - {e.read().decode()}")
        
        def generate_video(prompt, api_key, model="gemini-omni-flash-preview", aspect_ratio="16:9", duration=None, image_path=None, video_path=None, output_path="output.mp4", strip_audio=False, previous_interaction_id=None):
            """Creates an interaction with the video model and downloads the resulting video using the official google-genai SDK."""
            duration = parse_and_validate_duration(duration)
            input_parts = []
        
            # 1. Resolve and add image inputs (reference/start/end frames)
            if image_path:
                if isinstance(image_path, list):
                    for path in image_path:
                        img_uri, img_mime = resolve_or_upload_asset(path, "image/png", api_key)
                        input_parts.append({
                            "type": "image",
                            "uri": img_uri,
                            "mime_type": img_mime
                        })
                else:
                    img_uri, img_mime = resolve_or_upload_asset(image_path, "image/png", api_key)
                    input_parts.append({
                        "type": "image",
                        "uri": img_uri,
                        "mime_type": img_mime
                    })
        
            # 2. Resolve and add video inputs (for edits or extensions)
            if video_path:
                if isinstance(video_path, list):
                    for path in video_path:
                        vid_uri, vid_mime = resolve_or_upload_asset(path, "video/mp4", api_key, strip_audio=strip_audio)
                        input_parts.append({
                            "type": "video",
                            "uri": vid_uri,
                            "mime_type": vid_mime
                        })
                else:
                    vid_uri, vid_mime = resolve_or_upload_asset(video_path, "video/mp4", api_key, strip_audio=strip_audio)
                    input_parts.append({
                        "type": "video",
                        "uri": vid_uri,
                        "mime_type": vid_mime
                    })
        
            # 3. Add text prompt
            input_parts.append({
                "type": "text",
                "text": prompt
            })
        
            # Construct the config
            video_config = {
                "type": "video",
                "aspect_ratio": aspect_ratio,
                "delivery": "uri"
            }
            if duration:
                video_config["duration"] = duration
        
            print(f"\nSending generation request using official google-genai SDK and model '{model}'...")
            print(f"Prompt: '{prompt}' | Aspect Ratio: {aspect_ratio} | Duration: {duration}")
        
            # Initialize the client and call interactions.create
            client = genai.Client(api_key=api_key)
            try:
                interaction = client.interactions.create(
                    model=model,
                    input=input_parts,
                    response_format=video_config,
                    previous_interaction_id=previous_interaction_id
                )
            except Exception as e:
                raise RuntimeError(f"Error generating video via SDK: {e}")
        
            print(f"Generation complete for '{prompt}'! Processing response...")
        
            interaction_id = interaction.id
            if interaction_id:
                print(f"Interaction ID: {interaction_id}")
        
            output_video = interaction.output_video
            if not output_video or not output_video.uri:
                err_msg = f"No video content found in response for '{prompt}'."
                if video_path:
                    err_msg += (
                        "\nWARNING: IMPORTANT REGIONAL RESTRICTION: Uploading videos to use for video edits is "
                        "not available in the EEA, Switzerland, United Kingdom, and some US states."
                    )
                raise RuntimeError(f"{err_msg}\nResponse output_video field: {output_video}")
        
            video_uri = output_video.uri
            print(f"Generated video URI for '{prompt}': {video_uri}")
        
            # Download the final video
            download_video_file(video_uri, output_path, api_key)
        
        def run_job(job, api_key):
            """Runs a single generation job inside a thread pool, catching exceptions."""
            prompt = job.get("prompt")
            if not prompt:
                print("Warning: Skipping job with empty prompt.", file=sys.stderr)
                return {"job": job, "status": "SKIPPED", "error": "Empty prompt"}
        
            aspect_ratio = job.get("aspect_ratio", "16:9")
            duration = job.get("duration")
            image_path = job.get("image")
            video_path = job.get("video")
            output_path = job.get("output")
            model = job.get("model", "gemini-omni-flash-preview")
            strip_audio = job.get("strip_audio", False)
            previous_interaction_id = job.get("previous_interaction_id")
        
            if not output_path:
                output_path = f"media/output_{slugify(prompt)}.mp4"
        
            print(f"[Parallel] Dispatching: '{prompt}' (Output: {output_path})")
        
            try:
                generate_video(
                    prompt=prompt,
                    api_key=api_key,
                    model=model,
                    aspect_ratio=aspect_ratio,
                    duration=duration,
                    image_path=image_path,
                    video_path=video_path,
                    output_path=output_path,
                    strip_audio=strip_audio,
                    previous_interaction_id=previous_interaction_id
                )
                return {"job": job, "status": "SUCCESS", "output_path": output_path}
            except Exception as e:
                print(f"[Parallel] Failed: '{prompt}' - Error: {e}", file=sys.stderr)
                return {"job": job, "status": "FAILED", "error": str(e)}
        
        def main():
            parser = argparse.ArgumentParser(description="Generate and edit videos using Gemini Omni Flash model via google-genai SDK (supports parallel batch execution).")
            parser.add_argument("prompt", nargs="?", help="Text prompt / instruction for a single video generation")
            parser.add_argument("--image", action="append", help="Optional local image path or File API URI for referencing / image-to-video (can be specified multiple times)")
            parser.add_argument("--video", action="append", help="Optional local video path or File API URI for editing / extending (can be specified multiple times)")
            parser.add_argument("--aspect-ratio", default="16:9", choices=["16:9", "9:16"], help="Aspect ratio (default: 16:9)")
            parser.add_argument("--duration", type=argparse_duration_type, default=None, help="Video duration as an integer between 3 and 10 seconds (e.g., 5, 10). Default: None (API/Model decides, typically 10s or matches source)")
            parser.add_argument("--model", default="gemini-omni-flash-preview", help="Gemini Omni Flash video model ID (default: gemini-omni-flash-preview)")
            parser.add_argument("--output", help="Local output file path for single generation (default: media/output.mp4)")
            parser.add_argument("--strip-audio", "-a", action="store_true", help="Completely strip/disable audio stream from the input video(s) before uploading so Gemini Omni Flash can regenerate new audio from scratch")
            parser.add_argument("--previous-interaction-id", help="Optional Interaction ID of a previous generation for turn-by-turn editing")
            parser.add_argument("--api-key", help="Gemini API Key (overrides env)")
        
            # Parallel batch configuration options
            parser.add_argument("--batch", help="Path to a JSON file containing an array of generation jobs")
            parser.add_argument("--prompts-file", help="Path to a text file containing one prompt per line to run in parallel")
            parser.add_argument("--concurrency", type=int, default=3, help="Maximum number of concurrent executions (default: 3)")
        
            args = parser.parse_args()
        
            api_key = get_api_key(args)
            if not api_key:
                print("Error: API key is not set. Use --api-key or set GEMINI_API_KEY environment variable.", file=sys.stderr)
                sys.exit(1)
        
            # 1. Handle Batch JSON execution
            if args.batch:
                if not os.path.exists(args.batch):
                    print(f"Error: Batch JSON file '{args.batch}' not found.", file=sys.stderr)
                    sys.exit(1)
                try:
                    with safe_user_path(args.batch).open("r", encoding="utf-8") as f:
                        jobs = json.load(f)
                    if not isinstance(jobs, list):
                        print("Error: Batch JSON file must contain a list/array of job objects.", file=sys.stderr)
                        sys.exit(1)
                except Exception as e:
                    print(f"Error parsing Batch JSON: {e}", file=sys.stderr)
                    sys.exit(1)
        
                print(f"Loaded {len(jobs)} jobs from batch JSON. Running with concurrency={args.concurrency}...")
        
            # 2. Handle Prompts File execution
            elif args.prompts_file:
                if not os.path.exists(args.prompts_file):
                    print(f"Error: Prompts file '{args.prompts_file}' not found.", file=sys.stderr)
                    sys.exit(1)
        
                jobs = []
                with safe_user_path(args.prompts_file).open("r", encoding="utf-8") as f:
                    for line in f:
                        line = line.strip()
                        if line and not line.startswith("#"):
                            jobs.append({
                                "prompt": line,
                                "aspect_ratio": args.aspect_ratio,
                                "duration": args.duration,
                                "image": args.image,
                                "video": args.video,
                                "model": args.model,
                                "strip_audio": args.strip_audio,
                                "previous_interaction_id": args.previous_interaction_id
                            })
                print(f"Loaded {len(jobs)} prompts from text file. Running with concurrency={args.concurrency}...")
        
            # 3. Handle standard single prompt execution
            else:
                if not args.prompt:
                    parser.print_help()
                    sys.exit(1)
        
                output_path = args.output if args.output else "media/output.mp4"
                try:
                    generate_video(
                        prompt=args.prompt,
                        api_key=api_key,
                        model=args.model,
                        aspect_ratio=args.aspect_ratio,
                        duration=args.duration,
                        image_path=args.image,
                        video_path=args.video,
                        output_path=output_path,
                        strip_audio=args.strip_audio,
                        previous_interaction_id=args.previous_interaction_id
                    )
                    sys.exit(0)
                except Exception as e:
                    print(f"Error: Generation failed: {e}", file=sys.stderr)
                    sys.exit(1)
        
            # Parallel Execution Loop
            if not jobs:
                print("Warning: No valid jobs found to execute.")
                sys.exit(0)
        
            results = []
            with ThreadPoolExecutor(max_workers=args.concurrency) as executor:
                futures = {executor.submit(run_job, job, api_key): job for job in jobs}
                for future in as_completed(futures):
                    results.append(future.result())
        
            # Print Batch Results Summary
            print("\n" + "="*50)
            print("BATCH PARALLEL EXECUTION SUMMARY")
            print("="*50)
            success_count = sum(1 for r in results if r["status"] == "SUCCESS")
            failed_count = sum(1 for r in results if r["status"] == "FAILED")
            skipped_count = sum(1 for r in results if r["status"] == "SKIPPED")
        
            print(f"Total: {len(results)} | Success: {success_count} | Failed: {failed_count} | Skipped: {skipped_count}\n")
            for r in results:
                status_str = r["status"]
                prompt = r["job"].get("prompt")
                if r["status"] == "SUCCESS":
                    print(f"  [{status_str}] '{prompt}' -> {r['output_path']}")
                else:
                    print(f"  [{status_str}] '{prompt}' -> Error: {r.get('error')}")
            print("="*50)
        
            if failed_count > 0:
                sys.exit(1)
            sys.exit(0)
        
        if __name__ == "__main__":
            main()
        
      • inspect_video.py 6.2 KB
        #!/usr/bin/env python3
        import argparse
        import json
        import os
        import subprocess
        import sys
        
        def format_size(size_bytes):
            """Formats file size in bytes to a human-readable string."""
            try:
                size_bytes = int(size_bytes)
            except (ValueError, TypeError):
                return "Unknown size"
        
            for unit in ['B', 'KB', 'MB', 'GB']:
                if size_bytes < 1024.0:
                    return f"{size_bytes:.2f} {unit}"
                size_bytes /= 1024.0
            return f"{size_bytes:.2f} TB"
        
        def parse_fps(fps_str):
            """Parses fractional frame rates like '30/1' or '24000/1001' into floats."""
            if not fps_str:
                return "Unknown"
            if "/" in fps_str:
                try:
                    num, den = map(float, fps_str.split("/"))
                    if den != 0:
                        val = num / den
                        if val.is_integer():
                            return f"{int(val)} fps"
                        return f"{val:.2f} fps"
                except (ValueError, ZeroDivisionError):
                    pass
            try:
                val = float(fps_str)
                if val.is_integer():
                    return f"{int(val)} fps"
                return f"{val:.2f} fps"
            except ValueError:
                return fps_str
        
        def inspect_video(file_path, raw=False):
            """Runs ffprobe on the video file and returns parsed metadata dictionary."""
            if not os.path.exists(file_path):
                raise FileNotFoundError(f"File not found: {file_path}")
        
            # Check if ffprobe is available
            try:
                subprocess.run(["ffprobe", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)
            except (subprocess.SubprocessError, FileNotFoundError):
                raise RuntimeError("ffprobe is not installed or not found in system PATH.")
        
            cmd = [
                "ffprobe",
                "-v", "error",
                "-show_format",
                "-show_streams",
                "-of", "json",
                file_path
            ]
        
            result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=True)
            data = json.loads(result.stdout)
        
            if raw:
                return data
        
            # Extract format level details
            fmt = data.get("format", {})
            duration = fmt.get("duration")
            size_bytes = fmt.get("size")
            bitrate = fmt.get("bit_rate")
        
            # Format files size
            size_str = format_size(size_bytes) if size_bytes else "Unknown"
        
            # Parse duration
            try:
                duration_val = float(duration) if duration else 0.0
                duration_str = f"{duration_val:.2f}s"
            except ValueError:
                duration_str = "Unknown"
                duration_val = None
        
            # Parse bitrate
            try:
                bitrate_kbps = f"{int(float(bitrate) / 1000)} kbps" if bitrate else "Unknown"
            except ValueError:
                bitrate_kbps = "Unknown"
        
            video_streams = [s for s in data.get("streams", []) if s.get("codec_type") == "video"]
            audio_streams = [s for s in data.get("streams", []) if s.get("codec_type") == "audio"]
        
            has_video = len(video_streams) > 0
            has_audio = len(audio_streams) > 0
        
            video_info = {}
            if has_video:
                v = video_streams[0]
                width = v.get("width")
                height = v.get("height")
                codec = v.get("codec_name", "Unknown").upper()
                r_fps = parse_fps(v.get("r_frame_rate"))
                avg_fps = parse_fps(v.get("avg_frame_rate"))
        
                # Prefer r_frame_rate but fallback to avg
                fps = r_fps if r_fps != "0 fps" and r_fps != "Unknown" else avg_fps
        
                video_info = {
                    "resolution": f"{width}x{height}" if width and height else "Unknown",
                    "width": width,
                    "height": height,
                    "fps": fps,
                    "codec": codec,
                    "duration": v.get("duration")
                }
        
            audio_info = {}
            if has_audio:
                a = audio_streams[0]
                codec = a.get("codec_name", "Unknown").upper()
                channels = a.get("channels", "Unknown")
                sample_rate = a.get("sample_rate")
                sample_rate_khz = f"{float(sample_rate)/1000:.1f} kHz" if sample_rate else "Unknown"
        
                audio_info = {
                    "codec": codec,
                    "channels": channels,
                    "sample_rate": sample_rate_khz
                }
        
            return {
                "file_name": os.path.basename(file_path),
                "file_size": size_str,
                "size_bytes": size_bytes,
                "duration": duration_str,
                "duration_seconds": duration_val,
                "bitrate": bitrate_kbps,
                "has_video": has_video,
                "video": video_info,
                "has_audio": has_audio,
                "audio": audio_info
            }
        
        def print_terminal_report(info):
            """Prints an aligned terminal report."""
            print(f"\nVideo Inspection Report: {info['file_name']}")
            print("=" * 50)
            print(f"File Size   : {info['file_size']}")
            print(f"Duration    : {info['duration']}")
            print(f"Bitrate     : {info['bitrate']}")
        
            print("\nVideo Stream Details:")
            if info["has_video"]:
                v = info["video"]
                print(f"  * Resolution : {v['resolution']}")
                print(f"  * Frame Rate : {v['fps']}")
                print(f"  * Codec      : {v['codec']}")
            else:
                print("  * No Video Stream Found.")
        
            print("\nAudio Stream Details:")
            if info["has_audio"]:
                a = info["audio"]
                print("  * Status     : Audio Present")
                print(f"  * Codec      : {a['codec']}")
                print(f"  * Channels   : {a['channels']}")
                print(f"  * Sample Rate: {a['sample_rate']}")
            else:
                print("  * Status     : No Audio Stream Present")
            print()
        
        def main():
            parser = argparse.ArgumentParser(description="Inspect video details (duration, frame rate, resolution, audio presence) using ffprobe.")
            parser.add_argument("file", help="Path to the video file to inspect")
            parser.add_argument("--json", action="store_true", help="Output parsed summary in JSON format")
            parser.add_argument("--raw", action="store_true", help="Output raw unmodified ffprobe JSON data")
        
            args = parser.parse_args()
        
            try:
                if args.raw:
                    info = inspect_video(args.file, raw=True)
                    print(json.dumps(info, indent=2))
                else:
                    info = inspect_video(args.file, raw=False)
                    if args.json:
                        print(json.dumps(info, indent=2))
                    else:
                        print_terminal_report(info)
            except Exception as e:
                print(f"Error inspecting video: {e}", file=sys.stderr)
                sys.exit(1)
        
        if __name__ == "__main__":
            main()
        
      • prep_video.py 10.5 KB
        #!/usr/bin/env python3
        import argparse
        import os
        import subprocess
        import sys
        from inspect_video import inspect_video, format_size
        
        def parse_timecode(time_str, total_duration=None):
            """Parses a time string (seconds, MM:SS, HH:MM:SS, or 'last') into float seconds."""
            if not time_str:
                return 0.0
            time_str = time_str.strip().lower()
        
            if time_str == "last":
                if total_duration is None:
                    raise ValueError("Total duration is required to calculate 'last' starting point.")
                target_dur = 10.0
                if total_duration <= target_dur:
                    return 0.0
                return total_duration - target_dur
        
            if ":" in time_str:
                parts = time_str.split(":")
                if len(parts) == 2:  # MM:SS
                    m, s = map(float, parts)
                    return m * 60.0 + s
                elif len(parts) == 3:  # HH:MM:SS
                    h, m, s = map(float, parts)
                    return h * 3600.0 + m * 60.0 + s
                else:
                    raise ValueError(f"Invalid timecode format: '{time_str}'. Use HH:MM:SS or MM:SS.")
        
            try:
                return float(time_str)
            except ValueError:
                raise ValueError(f"Invalid timecode: '{time_str}'. Must be float seconds, HH:MM:SS, or 'last'.")
        
        def prep_video(input_path, output_path, start_time_str=None, duration=10, fps=None, resolution=None, strip_audio=False):
            """Preps a video file by trimming, optionally re-encoding to target fps and resolution."""
            if not os.path.exists(input_path):
                raise FileNotFoundError(f"Input file not found: {input_path}")
        
            # Check if ffmpeg is available
            try:
                subprocess.run(["ffmpeg", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)
            except (subprocess.SubprocessError, FileNotFoundError):
                raise RuntimeError("ffmpeg is not installed or not found in system PATH.")
        
            # Inspect input video first
            print(f"Analyzing source video: {os.path.basename(input_path)}...")
            source_info = inspect_video(input_path)
            total_duration = source_info.get("duration_seconds", 0.0)
        
            # Resolve start time
            if start_time_str is None:
                if total_duration and total_duration > 10.0 and sys.stdin.isatty():
                    print(f"\nThe input video is longer than 10s ({total_duration:.2f}s).")
                    print("Please choose a 10s segment to trim:")
                    print("  1) First 10 seconds [default]")
                    print("  2) Last 10 seconds")
                    print("  3) Custom starting timecode (e.g., MM:SS, HH:MM:SS, or seconds)")
                    try:
                        choice = input("Your choice [1/2/3, default 1]: ").strip()
                        if choice == "2":
                            start_time_str = "last"
                        elif choice == "3":
                            custom_start = input("Enter starting timecode (e.g., 00:03 or 15): ").strip()
                            start_time_str = custom_start if custom_start else "0"
                        else:
                            start_time_str = "0"
                    except (KeyboardInterrupt, EOFError):
                        print("\nNo input received. Defaulting to first 10 seconds.")
                        start_time_str = "0"
                else:
                    start_time_str = "0"
        
            try:
                start_seconds = parse_timecode(start_time_str, total_duration)
            except Exception as e:
                raise ValueError(f"Timecode parsing failed: {e}")
        
        
            if start_seconds < 0 or (total_duration and start_seconds >= total_duration):
                raise ValueError(f"Start time {start_seconds}s is out of bounds for video of length {total_duration}s.")
        
            # Construct output path if not specified
            if not output_path:
                os.makedirs("media", exist_ok=True)
                base_name = os.path.basename(input_path)
                name, ext = os.path.splitext(base_name)
                output_path = os.path.join("media", f"prepped_{name}.mp4")
        
            # Check if the source video file is large (>25MB)
            size_bytes_str = source_info.get("size_bytes")
            is_large = False
            try:
                if size_bytes_str and int(size_bytes_str) > 25 * 1024 * 1024:
                    is_large = True
            except (ValueError, TypeError):
                pass
        
            # Target resolution parsing
            scale_filter = None
            orig_width = None
            orig_height = None
            if "video" in source_info:
                try:
                    orig_width = int(source_info["video"].get("width"))
                    orig_height = int(source_info["video"].get("height"))
                except (ValueError, TypeError):
                    pass
        
            if resolution:
                try:
                    target_w, target_h = map(int, resolution.lower().split("x"))
                    if orig_width and orig_height:
                        # Scale to fit target_w and target_h while preserving aspect ratio
                        scale_factor = min(target_w / orig_width, target_h / orig_height)
                        width = int(orig_width * scale_factor)
                        height = int(orig_height * scale_factor)
                    else:
                        width, height = target_w, target_h
                    # Ensure divisible by 2 for standard decoders/encoders
                    width = (width // 2) * 2
                    height = (height // 2) * 2
                    scale_filter = f"scale={width}:{height}"
                    resolution = f"{width}x{height}"
                except ValueError:
                    raise ValueError(f"Invalid resolution: '{resolution}'. Format must be WIDTHxHEIGHT (e.g. 1280x720).")
            elif is_large:
                if orig_width and orig_height:
                    # Scale down large videos proportionally (max 1280x720 for landscape, 720x1280 for portrait)
                    if orig_width >= orig_height:
                        max_w, max_h = 1280, 720
                    else:
                        max_w, max_h = 720, 1280
                    scale_factor = min(max_w / orig_width, max_h / orig_height)
                    if scale_factor < 1.0:
                        width = int(orig_width * scale_factor)
                        height = int(orig_height * scale_factor)
                    else:
                        width, height = orig_width, orig_height
                else:
                    width, height = 1280, 720
        
                # Ensure divisible by 2
                width = (width // 2) * 2
                height = (height // 2) * 2
                resolution = f"{width}x{height}"
                print(f"\nRecommendation: Source video is very large ({source_info.get('file_size')}).")
                print("   Automatically scaling to optimize upload times for Gemini Omni Flash.")
                scale_filter = f"scale={width}:{height}"
        
            fps_spec = f"{fps} fps" if fps else "Original frame rate"
            print(f"\nPreparing Video Processing:")
            print(f"  * Source Duration: {total_duration:.2f}s")
            print(f"  * Trim Range     : Start at {start_seconds:.2f}s | Length {duration:.2f}s")
            if resolution:
                print(f"  * Encoding Specs : {width}x{height} @ {fps_spec}")
            else:
                print(f"  * Encoding Specs : Original Resolution @ {fps_spec}")
            print(f"  * Target Path    : {output_path}")
            print("=" * 50)
        
            # ffmpeg command construction
            cmd = [
                "ffmpeg",
                "-y",               # Overwrite output
                "-ss", str(start_seconds), # Seek start
                "-i", input_path,   # Input file
                "-t", str(duration), # Duration to copy
            ]
            if scale_filter:
                cmd.extend(["-vf", scale_filter])
        
            cmd.extend([
                "-c:v", "libx264",  # Standard H264 video codec
                "-pix_fmt", "yuv420p", # Standard pixel format for web/Gemini compatibility
            ])
        
            if fps:
                cmd.extend(["-r", str(fps)]) # Output frame rate if requested
        
            if strip_audio or not source_info.get("has_audio", False):
                if not source_info.get("has_audio", False) and not strip_audio:
                    print("No audio stream detected in source video. Disabling audio output.")
                else:
                    print("Stripping audio stream from video as requested.")
                cmd.append("-an")   # Disable audio streams completely
            else:
                cmd.extend([
                    "-c:a", "aac",      # Convert audio to standard AAC
                    "-b:a", "128k",     # Standard audio bitrate
                    "-ac", "2",         # Convert to stereo
                ])
        
            cmd.append(output_path)
        
            print("Running ffmpeg encoding...")
            process = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
        
            if process.returncode != 0:
                print("Error: ffmpeg failed. Stderr output follows:", file=sys.stderr)
                print(process.stderr, file=sys.stderr)
                raise RuntimeError("ffmpeg execution failed.")
        
            print("Video preparation completed successfully!")
            print("=" * 50)
        
            # Call inspection tool on output to print clean specs
            output_info = inspect_video(output_path)
            return output_info
        
        def main():
            parser = argparse.ArgumentParser(description="Prep videos for editing (trimming, re-encoding to target fps and resolution).")
            parser.add_argument("file", help="Path to the source video file to prep")
            parser.add_argument("--start", "-s", default=None, help="Start timecode (seconds, MM:SS, HH:MM:SS, or 'last' for last 10s). Default: 0 (or prompted if > 10s)")
            parser.add_argument("--duration", "-d", type=int, default=10, help="Duration of trimmed segment in seconds. Default: 10")
            parser.add_argument("--fps", "-r", type=int, default=None, help="Target frame rate. Default: None (keep original frame rate)")
            parser.add_argument("--resolution", "-g", default=None, help="Target resolution (e.g., 1280x720). Default: None (keep original resolution)")
            parser.add_argument("--output", "-o", help="Custom output path. Defaults to media/prepped_<original_name>.mp4")
            parser.add_argument("--strip-audio", "-a", action="store_true", help="Completely strip/disable audio stream so the model can generate new audio")
        
            args = parser.parse_args()
        
            try:
                info = prep_video(
                    input_path=args.file,
                    output_path=args.output,
                    start_time_str=args.start,
                    duration=args.duration,
                    fps=args.fps,
                    resolution=args.resolution,
                    strip_audio=args.strip_audio
                )
        
                # Display final output report
                print(f"\nPrepped Video Specifications: {info['file_name']}")
                print("=" * 50)
                print(f"File Size   : {info['file_size']}")
                print(f"Duration    : {info['duration']}")
                print(f"Bitrate     : {info['bitrate']}")
                print(f"Resolution  : {info['video']['resolution']}")
                print(f"Frame Rate  : {info['video']['fps']}")
                print(f"Video Codec : {info['video']['codec']}")
                if info["has_audio"]:
                    print(f"Audio Spec  : {info['audio']['codec']} | {info['audio']['channels']} ch | {info['audio']['sample_rate']}")
                print()
        
            except Exception as e:
                print(f"Error prepping video: {e}", file=sys.stderr)
                sys.exit(1)
        
        if __name__ == "__main__":
            main()
        
    • upload_file.py 9.2 KB
      #!/usr/bin/env python3
      """
      Uploads a file to the Gemini Files API and waits for it to become ACTIVE.
      Uses the official google-genai SDK.
      """
      
      import argparse
      import json
      import mimetypes
      import os
      import sys
      import time
      from google import genai
      from google.genai import types
      
      def get_api_key(args):
          """Retrieves API key from command args or environment."""
          if args.api_key:
              return args.api_key
          return os.environ.get("GEMINI_API_KEY")
      
      def detect_mime_type(file_path):
          """Determines MIME type based on file extension, falling back to standard mimetypes module."""
          ext = os.path.splitext(file_path)[1].lower()
          mime_map = {
              ".png": "image/png",
              ".jpg": "image/jpeg",
              ".jpeg": "image/jpeg",
              ".webp": "image/webp",
              ".mp4": "video/mp4",
              ".mp3": "audio/mpeg",
              ".wav": "audio/wav",
              ".pdf": "application/pdf",
              ".txt": "text/plain",
          }
      
          if ext in mime_map:
              return mime_map[ext]
      
          mime_type, _ = mimetypes.guess_type(file_path)
          if mime_type:
              return mime_type
      
          return "application/octet-stream"
      
      def upload_file(file_path, display_name=None, api_key=None):
          """Performs an upload using google-genai SDK, with automatic pre-processing for large videos."""
          file_size = os.path.getsize(file_path)
          mime_type = detect_mime_type(file_path)
      
          # Large video file size check (>25MB)
          is_video = mime_type.startswith("video/")
          if is_video and file_size > 25 * 1024 * 1024:
              size_mb = file_size / (1024 * 1024)
              print(f"\nWARNING: Video file '{file_path}' is very large ({size_mb:.2f} MB)!")
              print("Note: Gemini Omni Flash is optimized for 10s videos at 720p and 24fps. Uploading very large or")
              print("high-resolution videos will significantly increase upload times and may cause Out-Of-Memory (OOM) errors.")
      
              # Determine if terminal is interactive
              if sys.stdin.isatty():
                  print("\nWould you like to automatically pre-process this video first using prep_video.py?")
                  print("This will trim, scale, and optimize the video to ensure a fast, OOM-safe upload.")
                  try:
                      choice = input("Pre-process video? [Y/n]: ").strip().lower()
                      if choice in ("", "y", "yes"):
                          prepped_output_path = os.path.join("media", f"prepped_{os.path.basename(file_path)}")
                          os.makedirs("media", exist_ok=True)
      
                          # Resolve prep_video.py script path
                          import subprocess
                          prep_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "video", "prep_video.py")
                          if not os.path.exists(prep_script):
                              prep_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "prep_video.py")
      
                          cmd = [sys.executable, prep_script, file_path, "--output", prepped_output_path]
                          print(f"Running: {' '.join(cmd)}")
      
                          try:
                              result = subprocess.run(cmd)
                              if result.returncode == 0 and os.path.exists(prepped_output_path):
                                  file_path = prepped_output_path
                                  file_size = os.path.getsize(file_path)
                                  print(f"\nPre-processing completed successfully! Proceeding with upload of prepped video ({file_size / (1024*1024):.2f} MB)...")
                              else:
                                  raise RuntimeError("Error: Video pre-processing failed. Proceeding with original file upload is not recommended.")
                          except Exception as e:
                              raise RuntimeError(f"Error executing prep_video.py: {e}")
                      else:
                          proceed_choice = input("Do you want to proceed with uploading the original large video anyway? [y/N]: ").strip().lower()
                          if proceed_choice not in ("y", "yes"):
                              raise RuntimeError("Upload cancelled by user. Please pre-process the video manually first.")
                  except (KeyboardInterrupt, EOFError):
                      raise RuntimeError("\nNo input received. Upload cancelled to prevent OOM.")
              else:
                  # Non-interactive mode
                  if file_size > 100 * 1024 * 1024: # Block files larger than 100MB in non-interactive mode
                      err_msg = (
                          f"Error: Video file is extremely large ({size_mb:.2f} MB) and script is running in non-interactive mode.\n"
                          "To prevent Out-Of-Memory (OOM) errors, upload has been blocked.\n"
                          "Please pre-process the video first using prep_video.py."
                      )
                      raise RuntimeError(err_msg)
                  else:
                      print("Proceeding with upload in non-interactive mode...", file=sys.stderr)
      
          if not display_name:
              display_name = os.path.basename(file_path)
      
          print(f"Preparing upload of '{file_path}' ({file_size} bytes, type: {mime_type})...")
      
          # Step 1: Initialize Client
          client = genai.Client(api_key=api_key)
      
          # Step 2: Upload file using SDK
          print("Uploading file bytes using google-genai SDK...")
          try:
              config = types.UploadFileConfig(
                  display_name=display_name,
                  mime_type=mime_type,
              )
              file_obj = client.files.upload(file=file_path, config=config)
              # Convert Pydantic File model to dictionary with both camelCase and snake_case keys for compatibility
              file_dict = json.loads(file_obj.model_dump_json())
              # Add camelCase field for mimeType
              if "mime_type" in file_dict:
                  file_dict["mimeType"] = file_dict["mime_type"]
              return file_dict
          except Exception as e:
              raise RuntimeError(f"Error uploading file via SDK: {e}")
      
      def wait_for_active(file_name, api_key, poll_interval=3, max_attempts=30, backoff_factor=1.5, max_interval=30):
          """Polls the file status until state is ACTIVE or FAILED using exponential backoff via the SDK."""
          print(f"Waiting for file {file_name} to finish processing...")
      
          client = genai.Client(api_key=api_key)
          attempt = 0
          current_interval = poll_interval
          consecutive_errors = 0
          max_consecutive_errors = 5
      
          while attempt < max_attempts:
              try:
                  file_obj = client.files.get(name=file_name)
                  state = file_obj.state
      
                  # Extract state name as a string (handling both Enum and String)
                  state_str = state.name if hasattr(state, "name") else str(state)
      
                  # Reset consecutive errors on successful API response
                  consecutive_errors = 0
      
                  if state_str == "ACTIVE":
                      print("File is ACTIVE and ready for generations!")
                      file_dict = json.loads(file_obj.model_dump_json())
                      if "mime_type" in file_dict:
                          file_dict["mimeType"] = file_dict["mime_type"]
                      return file_dict
                  elif state_str == "FAILED":
                      raise RuntimeError("Error: File processing failed on the backend.")
      
                  print(f"Current state: {state_str}. Retrying in {current_interval:.1f}s...")
                  time.sleep(current_interval)
      
                  # Increase interval for the next poll (backoff)
                  current_interval = min(current_interval * backoff_factor, max_interval)
                  attempt += 1
      
              except Exception as e:
                  consecutive_errors += 1
                  if consecutive_errors >= max_consecutive_errors:
                      raise RuntimeError(f"Error: Too many consecutive errors checking status ({e}). Exiting.")
      
                  print(f"Warning: Error checking status ({e}). Retrying in {current_interval:.1f}s...")
                  time.sleep(current_interval)
                  current_interval = min(current_interval * backoff_factor, max_interval)
                  attempt += 1
      
          raise RuntimeError(f"Error: Maximum polling attempts ({max_attempts}) reached. File is still not ACTIVE.")
      
      def main():
          parser = argparse.ArgumentParser(description="Upload files to Gemini Files API using google-genai SDK.")
          parser.add_argument("file", help="Path to the file to upload")
          parser.add_argument("--name", help="Custom display name for the file")
          parser.add_argument("--api-key", help="Gemini API Key (overrides env)")
          parser.add_argument("--no-wait", action="store_true", help="Don't wait for ACTIVE status")
      
          args = parser.parse_args()
      
          api_key = get_api_key(args)
          if not api_key:
              print("Error: API key is not set. Use --api-key or set GEMINI_API_KEY environment variable.", file=sys.stderr)
              sys.exit(1)
      
          if not os.path.exists(args.file):
              print(f"Error: File '{args.file}' not found.", file=sys.stderr)
              sys.exit(1)
      
          try:
              file_meta = upload_file(args.file, args.name, api_key)
              file_name = file_meta.get("name")
      
              print(f"File metadata created:")
              print(f"  Name: {file_name}")
              print(f"  URI:  {file_meta.get('uri')}")
              print(f"  Type: {file_meta.get('mimeType')}")
      
              if not args.no_wait:
                  file_meta = wait_for_active(file_name, api_key)
      
              print("\nFile upload successfully completed! JSON Output:")
              print(json.dumps(file_meta, indent=2))
              sys.exit(0)
          except Exception as e:
              print(f"Error: {e}", file=sys.stderr)
              sys.exit(1)
      
      if __name__ == "__main__":
          main()
      
  • SKILL.md 16.5 KB
    ---
    name: gemini-omni-flash-api
    description: Use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,...
    risk: critical
    source: https://github.com/google-gemini/gemini-skills/tree/main/skills/gemini-omni-flash-api
    source_repo: google-gemini/gemini-skills
    source_type: official
    date_added: 2026-07-01
    license: Apache-2.0
    license_source: https://github.com/google-gemini/gemini-skills/blob/main/LICENSE
    ---
    
    # Gemini Omni Flash Skill
    ## When to Use
    
    Use this skill when you need use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,...
    
    
    This skill uses the Gemini Omni Flash model (`gemini-omni-flash-preview`) to perform text to video generation, image to video generation and video editing.
    
    > [!WARNING]
    > **Important Regional Restrictions**: Uploading videos to use for video edits is **NOT** available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (`total_output_tokens: 0` or no video content), it is likely due to this restriction.
    
    ## Core capabilities
    
    1. **Video editing and refinement**: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting.
    2. **Text to video**: Generating videos from a text prompt.
    3. **First-frame to video**: Generating videos from a single input image.
    4. **Image-referenced generation**: Using style, character, or object references from images to guide video generation.
    
    ## Workflow
    
    1. **Analyze request**: Determine the target task (e.g., first-frame-to-video, reference-guided editing) and identify any input media assets.
    2. **Run SDK scripts**:
    
       * Directly run the appropriate utility (`scripts/video/generate_video.py` or `scripts/upload_file.py`).
       * Configure settings like `--aspect-ratio` (e.g. `16:9`, `9:16`) and `--duration` (any integer between `3` and `10` seconds, e.g. `3`, `5`, `10`).
    
    3. **Retrieve and process output**: Outputs are saved to the local filesystem (e.g. `media/`). Report back the completed media path to the user.
    
    ## Reference Documentation
    
    * **Interactions API**: All operations and state management for the Gemini Omni Flash model (`gemini-omni-flash-preview`) are handled via the [Interactions API](https://ai.google.dev/gemini-api/docs/interactions-overview).
    * **Files API**: Input media files (such as reference images and videos) must be uploaded via the [Files API](https://ai.google.dev/gemini-api/docs/interactions/files) first before being referenced in generations. The uploaded file URI and MIME type are then included in the `interactions.create` input parts array.
    * **[Interactions API Skill Reference](https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-interactions-api/SKILL.md)**: Platform-wide guidelines, current model specifications, and SDK usage rules for the Interactions API.
    
    ## Dependencies and Prerequisites
    
    * **Python SDK (`google-genai`)**: Requires `google-genai >= 2.10.0` (Python) to support the new `interactions` client attribute. Install or upgrade using:
      ```bash
      pip install -U google-genai
      ```
    * **Python Runtime**: Requires **Python >= 3.10** (for compatibility with modern `google-genai` SDK types and methods).
    * **ffmpeg & ffprobe**: `prep_video.py`, `inspect_video.py`, and `generate_video.py` (when stripping audio via `--strip-audio`) require `ffmpeg` and `ffprobe` binaries installed and available in your system `PATH`.
    
    ## Available scripts
    
    Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API.
    
    1. **[upload_file.py](scripts/upload_file.py)**: Uploads local media (images and videos) to the Files API and polls until `ACTIVE`. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing with `prep_video.py` first to speed up the upload.
    
       ```bash
       ./scripts/upload_file.py path/to/image.png
       ```
    
    2. **[generate_video.py](scripts/video/generate_video.py)**: Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload.
    
       * **Text to video**:
    
         ```bash
         ./scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4
         ```
    
       * **Image to video (first frame and reference)**:
    
         ```bash
         ./scripts/video/generate_video.py "The waves crash against the shore." --image reference.png --output media/waves.mp4
         ```
    
       * **Video interpolation**:
    
         Provide exactly two images as keyframes to generate a transition video between them:
    
         ```bash
         ./scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --image start.png --image end.png --output media/interpolation.mp4
         ```
    
       * **Video editing (keep original audio)**:
    
         ```bash
         ./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --output media/anime_style.mp4
         ```
    
       * **Video editing (regenerate all audio from scratch)**:
    
         ```bash
         ./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --strip-audio --output media/anime_style_new_audio.mp4
         ```
    
       * **Turn-by-turn video editing (edit previous interaction)**:
    
         Edit a prior video generation without re-uploading assets by passing the interaction ID:
    
         ```bash
         ./scripts/video/generate_video.py "Change the setting to a snowy winter wonderland." --previous-interaction-id "abc123xyz..." --output media/winter_wonderland.mp4
         ```
    
       * **Parallel batch execution (prompts file)**: Run multiple prompts from a line-by-line text file concurrently:
    
         ```bash
         ./scripts/video/generate_video.py --prompts-file prompts.txt --concurrency 3
         ```
    
       * **Parallel batch execution (JSON config)**: Execute fully configured, distinct generation and editing jobs in parallel:
    
         ```bash
         ./scripts/video/generate_video.py --batch jobs.json --concurrency 3
         ```
    
         *Example `jobs.json`:*
    
         ```json
         [
           {
             "prompt": "Transform the style to Japanese anime.",
             "video": "input.mp4",
             "output": "media/anime_style.mp4",
             "strip_audio": false,
             "aspect_ratio": "16:9"
           },
           {
             "prompt": "A smooth timelapse from sunrise to sunset.",
             "image": ["start.png", "end.png"],
             "output": "media/interpolation.mp4"
           }
         ]
         ```
    
    3. **[inspect_video.py](scripts/video/inspect_video.py)**: Inspects a local video file (using `ffprobe`) to check its duration, resolution, frame rate (FPS), audio stream presence, and format details.
    
       ```bash
       ./scripts/video/inspect_video.py media/output.mp4
       ```
    
       * To get a pre-parsed, structured JSON summary:
    
         ```bash
         ./scripts/video/inspect_video.py media/output.mp4 --json
         ```
    
       * To get the complete, unmodified `ffprobe` raw JSON dump:
    
         ```bash
         ./scripts/video/inspect_video.py media/output.mp4 --raw
         ```
    
    4. **[prep_video.py](scripts/video/prep_video.py)**: Normalizes, trims, and formats any video file to fit standard Gemini Omni Flash generation and editing limits. It handles timecode-based trimming, optional frame rate conversion, and proportional scaling of large videos (max 1280x720 for landscape, 720x1280 for portrait) to optimize upload times without stretching. If the video is longer than 10 seconds and the script is run interactively (in a TTY), it prompts the user to select the first 10s, last 10s, or enter a custom timecode (defaulting to the first 10s).
    
       * **Trim first 10s (default)**:
    
        ```bash
         ./scripts/video/prep_video.py path/to/source.mp4
         ```
    
         or explicitly specify the start and duration:
    
         ```bash
         ./scripts/video/prep_video.py path/to/source.mp4 --start 0 --duration 10
         ```
    
       * **Trim last 10s** (automatically calculates starting point based on source length):
    
         ```bash
         ./scripts/video/prep_video.py path/to/source.mp4 --start last
         ```
    
       * **Trim 10s starting at specific timecode** (MM:SS or HH:MM:SS):
    
         ```bash
         ./scripts/video/prep_video.py path/to/source.mp4 --start 00:03 --output media/custom.mp4
         ```
    
       * **Custom frame rate and resolution**:
    
         ```bash
         ./scripts/video/prep_video.py path/to/source.mp4 --fps 30 --resolution 1920x1080
         ```
    
       * **Strip audio for audio regeneration**:
    
         ```bash
         ./scripts/video/prep_video.py path/to/source.mp4 --strip-audio --output media/video_with_no_audio.mp4
         ```
    
    ## Using tags in prompts to set image roles
    
    You can use tags in your prompt to make it clear whether each uploaded media is an initial frame or a reference.
    
    ### 1. Simple tags (recommended)
    
    For simple cases where image roles are clear from the prompt, you can bind images to roles directly:
    
    * **`<FIRST_FRAME>`**: Use the image as the starting frame of the video, for example: `<FIRST_FRAME> a woman is walking`
    * **`<IMAGE_REF_N>`**: Use the image as a reference, for example: `in the style of <IMAGE_REF_0> a woman <IMAGE_REF_1> is walking` (combines style reference from the first image and subject reference from the second image). Image references start from 0.
    
    An example with 6 reference images:
    
    ```none
    [0-3s] A studio fashion sequence. Starting with woman <IMAGE_REF_0>, she is holding <IMAGE_REF_1>
    [3-6s] Then we see the man <IMAGE_REF_2> holding <IMAGE_REF_3>
    [6-10s] And finally another woman <IMAGE_REF_4> who is holding <IMAGE_REF_5> while walking.
    ```
    
    ### 2. Explicitly declare sources and references
    
    For more complex cases with multiple images and multiple roles, you can use explicit prefix tags paired with natural language instruction suffixes.
    
    * **Declaring sources and reference images**:
      * `[# Sources <FIRST_FRAME>@Image1]` will use the first image as the starting frame.
      * `[# References <IMAGE_REF_0>@Image1]` will use the first image as a reference.
      * `[# References <IMAGE_REF_1>@Image2]` will use the second image as a reference.
      * `[# References <IMAGE_REF_0>@Image1 <IMAGE_REF_1>@Image2]` will use both images as references.
      * `[# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2]` will use the first image as the starting frame and the second image as a reference.
    * **Guiding instructions**: Add guiding instructions at the end of your prompt:
      * For starting frame: `"Use the given image as the starting frame."`
      * For reference images: `"Use the given image(s) as references for video generation. The images should not be used as literal initial frames."`
    
    * *Example Expanded Prompt*:
    
      ```none
      [# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2] a woman <IMAGE_REF_0> is walking. Use Image1 as the starting frame. Use Image2 as a reference for the video generation.
      ```
    
    ## Audio handling in video editing
    
    When editing a source video that contains audio, you must choose between keeping the original audio or regenerating all audio from scratch.
    
    * **Keep original audio**: By default, Gemini Omni Flash preserves the existing audio layer (though it may modify or adapt it slightly during generation). Use this when the original background music, dialogue, or sound effects are desired.
    * **Regenerate all audio from scratch**: If you want Gemini Omni Flash to re-create a brand-new audio layer tailored to the new visual style or prompt, you **must** upload the video with its audio stream stripped out. If any audio stream is present, Gemini Omni Flash will attempt to preserve/modify it instead of starting from scratch.
    
      * Use `--strip-audio` (or `-a`) when pre-processing with `scripts/video/prep_video.py` or executing `scripts/video/generate_video.py`.
      * This forces Gemini Omni Flash to perform full audio generation.
    
    ## Prompting Gemini Omni Flash
    
    ### Single scene
    
    By default Gemini Omni Flash will try to create a video with a few different shots. It'll attempt to craft an interesting narrative based on the prompt.
    
    If you need the output video to contain a single scene, you must prompt for that:
    
    * In a single unbroken scene
    * In a single continuous shot
    * No scene cuts
    
    For example:
    
    ```none
    Continuous, unbroken handheld shot of a fluffy tabby cat sitting on a sunny windowsill, looking out into a leafy garden. The cat's tail twitches slowly, and its ears rotate slightly toward ambient noises. Sunbeams illuminate dust motes in the air. Sound design: Gentle breeze, distant bird chirps, quiet mechanical purring. No dialogue.
    ```
    
    ### Removing unwanted elements
    
    If generations contain things you don't want, you can include simple negatives to avoid them:
    
    * No dialogue
    * No embellishments
    * No extra sound effects
    
    ### Prompts for editing
    
    Simple prompts work best for editing. Overly descriptive prompts can lead to unintended changes.
    
    For example:
    
    * Make this video anime
    * Make the phone invisible
    * Put a fashionable hat on this person
    * Change the lighting to be more dramatic
    * Change the text on the sign to say "Gemini Omni Flash"
    * Add a cat that jumps onto his lap, he begins to pet it
    
    When editing a specific aspect of the video, it can help to include: "Keep everything else the same".
    
    ### Prompting the audio
    
    By default the model will try to generate an appropriate audio track for a video. This might not always be what you want. You can use your prompt to describe the type of audio you want. This is especially important if you want music in your video:
    
    * Include calm background music
    * The video has a high energy techno beat
    * The audio is a low tinny radio broadcast in the background, playing a song
    * Audio design: [a description of the audio you want]
    
    ### When things should happen
    
    You can prompt for things to happen at specific times in the video, there is no precise syntax needed and you can use natural language. This is especially useful in creating your own scene cuts, rhythm or rapid fire sequences.
    
    Simple examples:
    
    * after 3 seconds, a woman enters the scene
    * at 5s the chorus starts in the background audio
    * every 2s cut to a new frame
    * in a rapid fire sequence, every half a second (12 frames at 24fps) change the scene to a new location
    
    You can also use a timecode syntax:
    
    ```none
    [0-3s] A person is walking
    [3-6s] They stop and turn around
    [6-10s] They start running
    ```
    
    ### Meta prompting
    
    Rather than specifying everything directly in a prompt, you can ask the model to pay attention to certain things. You can give Gemini Omni Flash these sorts of prompts verbatim:
    
    * Consider micro-detail, expression and timing to create a very rich, detailed but entirely natural scene.
    * Be extremely detailed in your descriptions of characters and environments. Apply costume design principles to characters. Be very specific about the people, items and objects in the scene.
    * Include plenty of appropriate detail in the background elements to make the scene feel realistic and natural.
    * Make a rapid fire video that shows a different rare [thing] every 1s, upbeat music, include text to label the thing.
    
    ### Text in videos works really well
    
    Unlike previous video models, text in Gemini Omni Flash videos works really well. You can include decent amounts of text in your video and it will be rendered in a way that is correct and readable. If there will be naturally occurring text in your video, even in background elements, it can help to define what it should say.
    
    For example:
    
    * One word on the screen at a time: "did, you, know, that, Omni, can, do, awesome, text?" Each word appears for 1s with a different animated style. No dialogue.
    * There is a street sign that says: "This is an AI generation by Omni", there is a storefront that says: "All you need AI", there's a car with the number plate: "OMN111"
    
    ## Limitations
    
    - Use this skill only when the task clearly matches its upstream product or API scope.
    - Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
    - Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related