gemini-omni-flash-api
Use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,..
Install
npx skills add https://github.com/sickn33/agentic-awesome-skills/tree/main/skills/gemini-omni-flash-api
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install sickn33-agentic-awesome-skills@llmmart
git clone https://github.com/sickn33/agentic-awesome-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole sickn33/agentic-awesome-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Gemini Omni Flash Skill
When to Use
Use this skill when you need use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,...
This skill uses the Gemini Omni Flash model (gemini-omni-flash-preview) to perform text to video generation, image to video generation and video editing.
Warning
Important Regional Restrictions: Uploading videos to use for video edits is NOT available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (total_output_tokens: 0 or no video content), it is likely due to this restriction.
Core capabilities
- Video editing and refinement: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting.
- Text to video: Generating videos from a text prompt.
- First-frame to video: Generating videos from a single input image.
- Image-referenced generation: Using style, character, or object references from images to guide video generation.
Workflow
Analyze request: Determine the target task (e.g., first-frame-to-video, reference-guided editing) and identify any input media assets.
Run SDK scripts:
- Directly run the appropriate utility (
scripts/video/generate_video.pyorscripts/upload_file.py). - Configure settings like
--aspect-ratio(e.g.16:9,9:16) and--duration(any integer between3and10seconds, e.g.3,5,10).
- Directly run the appropriate utility (
Retrieve and process output: Outputs are saved to the local filesystem (e.g.
media/). Report back the completed media path to the user.
Reference Documentation
- Interactions API: All operations and state management for the Gemini Omni Flash model (
gemini-omni-flash-preview) are handled via the Interactions API. - Files API: Input media files (such as reference images and videos) must be uploaded via the Files API first before being referenced in generations. The uploaded file URI and MIME type are then included in the
interactions.createinput parts array. - Interactions API Skill Reference: Platform-wide guidelines, current model specifications, and SDK usage rules for the Interactions API.
Dependencies and Prerequisites
- Python SDK (
google-genai): Requiresgoogle-genai >= 2.10.0(Python) to support the newinteractionsclient attribute. Install or upgrade using:pip install -U google-genai - Python Runtime: Requires Python >= 3.10 (for compatibility with modern
google-genaiSDK types and methods). - ffmpeg & ffprobe:
prep_video.py,inspect_video.py, andgenerate_video.py(when stripping audio via--strip-audio) requireffmpegandffprobebinaries installed and available in your systemPATH.
Available scripts
Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API.
upload_file.py: Uploads local media (images and videos) to the Files API and polls until
ACTIVE. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing withprep_video.pyfirst to speed up the upload../scripts/upload_file.py path/to/image.pnggenerate_video.py: Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload.
Text to video:
./scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4Image to video (first frame and reference):
./scripts/video/generate_video.py "The waves crash against the shore." --image reference.png --output media/waves.mp4Video interpolation:
Provide exactly two images as keyframes to generate a transition video between them:
./scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --image start.png --image end.png --output media/interpolation.mp4Video editing (keep original audio):
./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --output media/anime_style.mp4Video editing (regenerate all audio from scratch):
./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --strip-audio --output media/anime_style_new_audio.mp4Turn-by-turn video editing (edit previous interaction):
Edit a prior video generation without re-uploading assets by passing the interaction ID:
./scripts/video/generate_video.py "Change the setting to a snowy winter wonderland." --previous-interaction-id "abc123xyz..." --output media/winter_wonderland.mp4Parallel batch execution (prompts file): Run multiple prompts from a line-by-line text file concurrently:
./scripts/video/generate_video.py --prompts-file prompts.txt --concurrency 3Parallel batch execution (JSON config): Execute fully configured, distinct generation and editing jobs in parallel:
./scripts/video/generate_video.py --batch jobs.json --concurrency 3Example
jobs.json:[ { "prompt": "Transform the style to Japanese anime.", "video": "input.mp4", "output": "media/anime_style.mp4", "strip_audio": false, "aspect_ratio": "16:9" }, { "prompt": "A smooth timelapse from sunrise to sunset.", "image": ["start.png", "end.png"], "output": "media/interpolation.mp4" } ]
inspect_video.py: Inspects a local video file (using
ffprobe) to check its duration, resolution, frame rate (FPS), audio stream presence, and format details../scripts/video/inspect_video.py media/output.mp4To get a pre-parsed, structured JSON summary:
./scripts/video/inspect_video.py media/output.mp4 --jsonTo get the complete, unmodified
ffproberaw JSON dump:./scripts/video/inspect_video.py media/output.mp4 --raw
prep_video.py: Normalizes, trims, and formats any video file to fit standard Gemini Omni Flash generation and editing limits. It handles timecode-based trimming, optional frame rate conversion, and proportional scaling of large videos (max 1280x720 for landscape, 720x1280 for portrait) to optimize upload times without stretching. If the video is longer than 10 seconds and the script is run interactively (in a TTY), it prompts the user to select the first 10s, last 10s, or enter a custom timecode (defaulting to the first 10s).
- Trim first 10s (default):
./scripts/video/prep_video.py path/to/source.mp4or explicitly specify the start and duration:
./scripts/video/prep_video.py path/to/source.mp4 --start 0 --duration 10Trim last 10s (automatically calculates starting point based on source length):
./scripts/video/prep_video.py path/to/source.mp4 --start lastTrim 10s starting at specific timecode (MM:SS or HH:MM:SS):
./scripts/video/prep_video.py path/to/source.mp4 --start 00:03 --output media/custom.mp4Custom frame rate and resolution:
./scripts/video/prep_video.py path/to/source.mp4 --fps 30 --resolution 1920x1080Strip audio for audio regeneration:
./scripts/video/prep_video.py path/to/source.mp4 --strip-audio --output media/video_with_no_audio.mp4
Using tags in prompts to set image roles
You can use tags in your prompt to make it clear whether each uploaded media is an initial frame or a reference.
1. Simple tags (recommended)
For simple cases where image roles are clear from the prompt, you can bind images to roles directly:
<FIRST_FRAME>: Use the image as the starting frame of the video, for example:<FIRST_FRAME> a woman is walking<IMAGE_REF_N>: Use the image as a reference, for example:in the style of <IMAGE_REF_0> a woman <IMAGE_REF_1> is walking(combines style reference from the first image and subject reference from the second image). Image references start from 0.
An example with 6 reference images:
[0-3s] A studio fashion sequence. Starting with woman <IMAGE_REF_0>, she is holding <IMAGE_REF_1>
[3-6s] Then we see the man <IMAGE_REF_2> holding <IMAGE_REF_3>
[6-10s] And finally another woman <IMAGE_REF_4> who is holding <IMAGE_REF_5> while walking.
2. Explicitly declare sources and references
For more complex cases with multiple images and multiple roles, you can use explicit prefix tags paired with natural language instruction suffixes.
Declaring sources and reference images:
[# Sources <FIRST_FRAME>@Image1]will use the first image as the starting frame.[# References <IMAGE_REF_0>@Image1]will use the first image as a reference.[# References <IMAGE_REF_1>@Image2]will use the second image as a reference.[# References <IMAGE_REF_0>@Image1 <IMAGE_REF_1>@Image2]will use both images as references.[# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2]will use the first image as the starting frame and the second image as a reference.
Guiding instructions: Add guiding instructions at the end of your prompt:
- For starting frame:
"Use the given image as the starting frame." - For reference images:
"Use the given image(s) as references for video generation. The images should not be used as literal initial frames."
- For starting frame:
Example Expanded Prompt:
[# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2] a woman <IMAGE_REF_0> is walking. Use Image1 as the starting frame. Use Image2 as a reference for the video generation.
Audio handling in video editing
When editing a source video that contains audio, you must choose between keeping the original audio or regenerating all audio from scratch.
Keep original audio: By default, Gemini Omni Flash preserves the existing audio layer (though it may modify or adapt it slightly during generation). Use this when the original background music, dialogue, or sound effects are desired.
Regenerate all audio from scratch: If you want Gemini Omni Flash to re-create a brand-new audio layer tailored to the new visual style or prompt, you must upload the video with its audio stream stripped out. If any audio stream is present, Gemini Omni Flash will attempt to preserve/modify it instead of starting from scratch.
- Use
--strip-audio(or-a) when pre-processing withscripts/video/prep_video.pyor executingscripts/video/generate_video.py. - This forces Gemini Omni Flash to perform full audio generation.
- Use
Prompting Gemini Omni Flash
Single scene
By default Gemini Omni Flash will try to create a video with a few different shots. It'll attempt to craft an interesting narrative based on the prompt.
If you need the output video to contain a single scene, you must prompt for that:
- In a single unbroken scene
- In a single continuous shot
- No scene cuts
For example:
Continuous, unbroken handheld shot of a fluffy tabby cat sitting on a sunny windowsill, looking out into a leafy garden. The cat's tail twitches slowly, and its ears rotate slightly toward ambient noises. Sunbeams illuminate dust motes in the air. Sound design: Gentle breeze, distant bird chirps, quiet mechanical purring. No dialogue.
Removing unwanted elements
If generations contain things you don't want, you can include simple negatives to avoid them:
- No dialogue
- No embellishments
- No extra sound effects
Prompts for editing
Simple prompts work best for editing. Overly descriptive prompts can lead to unintended changes.
For example:
- Make this video anime
- Make the phone invisible
- Put a fashionable hat on this person
- Change the lighting to be more dramatic
- Change the text on the sign to say "Gemini Omni Flash"
- Add a cat that jumps onto his lap, he begins to pet it
When editing a specific aspect of the video, it can help to include: "Keep everything else the same".
Prompting the audio
By default the model will try to generate an appropriate audio track for a video. This might not always be what you want. You can use your prompt to describe the type of audio you want. This is especially important if you want music in your video:
- Include calm background music
- The video has a high energy techno beat
- The audio is a low tinny radio broadcast in the background, playing a song
- Audio design: [a description of the audio you want]
When things should happen
You can prompt for things to happen at specific times in the video, there is no precise syntax needed and you can use natural language. This is especially useful in creating your own scene cuts, rhythm or rapid fire sequences.
Simple examples:
- after 3 seconds, a woman enters the scene
- at 5s the chorus starts in the background audio
- every 2s cut to a new frame
- in a rapid fire sequence, every half a second (12 frames at 24fps) change the scene to a new location
You can also use a timecode syntax:
[0-3s] A person is walking
[3-6s] They stop and turn around
[6-10s] They start running
Meta prompting
Rather than specifying everything directly in a prompt, you can ask the model to pay attention to certain things. You can give Gemini Omni Flash these sorts of prompts verbatim:
- Consider micro-detail, expression and timing to create a very rich, detailed but entirely natural scene.
- Be extremely detailed in your descriptions of characters and environments. Apply costume design principles to characters. Be very specific about the people, items and objects in the scene.
- Include plenty of appropriate detail in the background elements to make the scene feel realistic and natural.
- Make a rapid fire video that shows a different rare [thing] every 1s, upbeat music, include text to label the thing.
Text in videos works really well
Unlike previous video models, text in Gemini Omni Flash videos works really well. You can include decent amounts of text in your video and it will be rendered in a way that is correct and readable. If there will be naturally occurring text in your video, even in background elements, it can help to define what it should say.
For example:
- One word on the screen at a time: "did, you, know, that, Omni, can, do, awesome, text?" Each word appears for 1s with a different animated style. No dialogue.
- There is a street sign that says: "This is an AI generation by Omni", there is a storefront that says: "All you need AI", there's a car with the number plate: "OMN111"
Limitations
- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
Files (agentic-awesome-skills)
-
scripts
-
video
-
generate_video.py 19.5 KB
#!/usr/bin/env python3 """ Generates and edits videos using the Gemini Omni Flash model via the google-genai Interactions API. Can automatically upload local media references using the Files API. Supports parallel execution of multiple generations using Python standard library. Uses the official google-genai SDK. """ import argparse from concurrent.futures import ThreadPoolExecutor, as_completed import json import os import re import sys import time import urllib.request import urllib.error import urllib.parse import uuid from google import genai # Load local upload helper logic inline to prevent dependency issues sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) from upload_file import upload_file, wait_for_active from pathlib import Path def safe_user_path(path_value, base_dir="."): """Resolve a CLI path under the current workspace.""" if base_dir != ".": raise ValueError("Custom base directories are not supported for CLI paths") base_path = Path.cwd().resolve() resolved_path = Path(path_value).expanduser().resolve() try: resolved_path.relative_to(base_path) except ValueError as exc: raise ValueError(f"Path escapes allowed directory: {path_value}") from exc return resolved_path def get_api_key(args): """Retrieves API key from command args or environment.""" if args.api_key: return args.api_key return os.environ.get("GEMINI_API_KEY") FILE_ID_RE = re.compile(r'^[A-Za-z0-9_-]+$') def extract_file_id(uri): """Returns a Gemini File API id from a local reference or trusted API URL.""" if not uri: return None if uri.startswith("files/"): file_id = uri.removeprefix("files/") return file_id if FILE_ID_RE.fullmatch(file_id) else None parsed = urllib.parse.urlparse(uri) if parsed.scheme != "https" or parsed.netloc != "generativelanguage.googleapis.com": return None path_match = re.fullmatch(r'/files/([A-Za-z0-9_-]+)', parsed.path) return path_match.group(1) if path_match else None def is_file_uri(uri): """Returns True if the string is a standard Gemini File URI.""" return extract_file_id(uri) is not None def normalize_file_uri(uri): """Normalizes any File API URI/reference to the standard https://generativelanguage.googleapis.com/files/{id} format.""" file_id = extract_file_id(uri) if file_id: return f"https://generativelanguage.googleapis.com/files/{file_id}" return uri def media_download_url(file_uri): """Build a media URL only for validated Gemini File API references.""" file_id = extract_file_id(file_uri) if not file_id: raise ValueError("Generated video URI must be a Gemini File API reference.") return f"https://generativelanguage.googleapis.com/files/{file_id}?alt=media" def slugify(text): """Converts a text prompt into a safe, descriptive filename slug.""" text = text.lower() text = re.sub(r'[^a-z0-9]+', '_', text) return text.strip('_')[:50] def parse_and_validate_duration(value): """Parses and formats a duration integer between 3 and 10 with optional 's' suffix.""" if value is None: return None if isinstance(value, (int, float)): val = float(value) else: clean_value = str(value).strip().lower() if clean_value in ('none', ''): return None if clean_value.endswith('s'): clean_value = clean_value[:-1] try: val = float(clean_value) except ValueError: raise ValueError(f"Invalid duration value: '{value}'. Must be an integer (e.g., 5, 10).") if not val.is_integer(): raise ValueError(f"Duration must be an integer, not a float (e.g., got {value}).") val_int = int(val) if val_int < 3 or val_int > 10: raise ValueError(f"Duration must be between 3 (inclusive) and 10 (inclusive) seconds. Got {val_int}.") return f"{val_int}s" def argparse_duration_type(value): """argparse type converter for validating duration.""" if value is None or str(value).strip().lower() in ('none', ''): return None try: return parse_and_validate_duration(value) except ValueError as e: raise argparse.ArgumentTypeError(str(e)) def resolve_or_upload_asset(asset_path, mime_type, api_key, strip_audio=False): """ If asset_path is a File API URI, returns it directly (normalized). If it is a local file path, uploads it and returns its File API URI (normalized). """ if not asset_path: return None, None if is_file_uri(asset_path): normalized = normalize_file_uri(asset_path) print(f"Using existing File URI: {normalized}") if strip_audio: print("Warning: --strip-audio was specified but the video input is an existing remote File URI. " "Audio cannot be stripped from remote files automatically.") return normalized, mime_type if os.path.exists(asset_path): upload_path = asset_path temp_stripped_path = None if strip_audio: print(f"Detected local asset path '{asset_path}'. Stripping audio before upload...") # Check if ffmpeg is available import subprocess try: subprocess.run(["ffmpeg", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) except (subprocess.SubprocessError, FileNotFoundError): raise RuntimeError( "Error: ffmpeg is not installed or not found in system PATH. " "ffmpeg is required to strip audio from local videos." ) try: os.makedirs("media", exist_ok=True) base_name = os.path.basename(asset_path) name, ext = os.path.splitext(base_name) temp_stripped_path = os.path.join("media", f"temp_stripped_{name}_{uuid.uuid4().hex}{ext}") # Fast stream-copy audio stripping cmd = ["ffmpeg", "-y", "-i", asset_path, "-c:v", "copy", "-an", temp_stripped_path] subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True) print(f"Successfully stripped audio. Temporary video file created at: {temp_stripped_path}") upload_path = temp_stripped_path except Exception as e: print(f"Error stripping audio using ffmpeg: {e}", file=sys.stderr) print("Falling back to uploading the original video with audio.", file=sys.stderr) print(f"Uploading asset '{upload_path}'...") file_meta = upload_file(upload_path, api_key=api_key) file_name = file_meta.get("name") # Wait for file to become active file_meta = wait_for_active(file_name, api_key) normalized = normalize_file_uri(file_meta.get("uri")) # Clean up temporary stripped file if we created one if temp_stripped_path and os.path.exists(temp_stripped_path): try: safe_user_path(temp_stripped_path).unlink() print(f"Cleaned up temporary video file: {temp_stripped_path}") except Exception as e: print(f"Warning: Failed to remove temporary file {temp_stripped_path}: {e}", file=sys.stderr) # Handle both mimeType and mime_type key formats returned from upload_file returned_mime = file_meta.get("mimeType") or file_meta.get("mime_type") return normalized, returned_mime else: raise FileNotFoundError(f"Asset path '{asset_path}' is neither a valid File API URI nor a local file path.") def download_video_file(file_uri, output_path, api_key): """Downloads generated video file from URI using alt=media standard in a memory-safe, chunked manner.""" download_url = media_download_url(file_uri) print(f"Downloading video from {file_uri} to {output_path} in chunked mode...") req = urllib.request.Request(download_url) req.add_header("x-goog-api-key", api_key) try: with urllib.request.urlopen(req, timeout=480) as resp: parent_dir = os.path.dirname(output_path) if parent_dir: os.makedirs(parent_dir, exist_ok=True) with safe_user_path(output_path).open("wb") as f: while True: chunk = resp.read(8192) if not chunk: break f.write(chunk) print(f"Video successfully saved to: {output_path}") except urllib.error.HTTPError as e: raise RuntimeError(f"Error downloading video file: {e.code} - {e.read().decode()}") def generate_video(prompt, api_key, model="gemini-omni-flash-preview", aspect_ratio="16:9", duration=None, image_path=None, video_path=None, output_path="output.mp4", strip_audio=False, previous_interaction_id=None): """Creates an interaction with the video model and downloads the resulting video using the official google-genai SDK.""" duration = parse_and_validate_duration(duration) input_parts = [] # 1. Resolve and add image inputs (reference/start/end frames) if image_path: if isinstance(image_path, list): for path in image_path: img_uri, img_mime = resolve_or_upload_asset(path, "image/png", api_key) input_parts.append({ "type": "image", "uri": img_uri, "mime_type": img_mime }) else: img_uri, img_mime = resolve_or_upload_asset(image_path, "image/png", api_key) input_parts.append({ "type": "image", "uri": img_uri, "mime_type": img_mime }) # 2. Resolve and add video inputs (for edits or extensions) if video_path: if isinstance(video_path, list): for path in video_path: vid_uri, vid_mime = resolve_or_upload_asset(path, "video/mp4", api_key, strip_audio=strip_audio) input_parts.append({ "type": "video", "uri": vid_uri, "mime_type": vid_mime }) else: vid_uri, vid_mime = resolve_or_upload_asset(video_path, "video/mp4", api_key, strip_audio=strip_audio) input_parts.append({ "type": "video", "uri": vid_uri, "mime_type": vid_mime }) # 3. Add text prompt input_parts.append({ "type": "text", "text": prompt }) # Construct the config video_config = { "type": "video", "aspect_ratio": aspect_ratio, "delivery": "uri" } if duration: video_config["duration"] = duration print(f"\nSending generation request using official google-genai SDK and model '{model}'...") print(f"Prompt: '{prompt}' | Aspect Ratio: {aspect_ratio} | Duration: {duration}") # Initialize the client and call interactions.create client = genai.Client(api_key=api_key) try: interaction = client.interactions.create( model=model, input=input_parts, response_format=video_config, previous_interaction_id=previous_interaction_id ) except Exception as e: raise RuntimeError(f"Error generating video via SDK: {e}") print(f"Generation complete for '{prompt}'! Processing response...") interaction_id = interaction.id if interaction_id: print(f"Interaction ID: {interaction_id}") output_video = interaction.output_video if not output_video or not output_video.uri: err_msg = f"No video content found in response for '{prompt}'." if video_path: err_msg += ( "\nWARNING: IMPORTANT REGIONAL RESTRICTION: Uploading videos to use for video edits is " "not available in the EEA, Switzerland, United Kingdom, and some US states." ) raise RuntimeError(f"{err_msg}\nResponse output_video field: {output_video}") video_uri = output_video.uri print(f"Generated video URI for '{prompt}': {video_uri}") # Download the final video download_video_file(video_uri, output_path, api_key) def run_job(job, api_key): """Runs a single generation job inside a thread pool, catching exceptions.""" prompt = job.get("prompt") if not prompt: print("Warning: Skipping job with empty prompt.", file=sys.stderr) return {"job": job, "status": "SKIPPED", "error": "Empty prompt"} aspect_ratio = job.get("aspect_ratio", "16:9") duration = job.get("duration") image_path = job.get("image") video_path = job.get("video") output_path = job.get("output") model = job.get("model", "gemini-omni-flash-preview") strip_audio = job.get("strip_audio", False) previous_interaction_id = job.get("previous_interaction_id") if not output_path: output_path = f"media/output_{slugify(prompt)}.mp4" print(f"[Parallel] Dispatching: '{prompt}' (Output: {output_path})") try: generate_video( prompt=prompt, api_key=api_key, model=model, aspect_ratio=aspect_ratio, duration=duration, image_path=image_path, video_path=video_path, output_path=output_path, strip_audio=strip_audio, previous_interaction_id=previous_interaction_id ) return {"job": job, "status": "SUCCESS", "output_path": output_path} except Exception as e: print(f"[Parallel] Failed: '{prompt}' - Error: {e}", file=sys.stderr) return {"job": job, "status": "FAILED", "error": str(e)} def main(): parser = argparse.ArgumentParser(description="Generate and edit videos using Gemini Omni Flash model via google-genai SDK (supports parallel batch execution).") parser.add_argument("prompt", nargs="?", help="Text prompt / instruction for a single video generation") parser.add_argument("--image", action="append", help="Optional local image path or File API URI for referencing / image-to-video (can be specified multiple times)") parser.add_argument("--video", action="append", help="Optional local video path or File API URI for editing / extending (can be specified multiple times)") parser.add_argument("--aspect-ratio", default="16:9", choices=["16:9", "9:16"], help="Aspect ratio (default: 16:9)") parser.add_argument("--duration", type=argparse_duration_type, default=None, help="Video duration as an integer between 3 and 10 seconds (e.g., 5, 10). Default: None (API/Model decides, typically 10s or matches source)") parser.add_argument("--model", default="gemini-omni-flash-preview", help="Gemini Omni Flash video model ID (default: gemini-omni-flash-preview)") parser.add_argument("--output", help="Local output file path for single generation (default: media/output.mp4)") parser.add_argument("--strip-audio", "-a", action="store_true", help="Completely strip/disable audio stream from the input video(s) before uploading so Gemini Omni Flash can regenerate new audio from scratch") parser.add_argument("--previous-interaction-id", help="Optional Interaction ID of a previous generation for turn-by-turn editing") parser.add_argument("--api-key", help="Gemini API Key (overrides env)") # Parallel batch configuration options parser.add_argument("--batch", help="Path to a JSON file containing an array of generation jobs") parser.add_argument("--prompts-file", help="Path to a text file containing one prompt per line to run in parallel") parser.add_argument("--concurrency", type=int, default=3, help="Maximum number of concurrent executions (default: 3)") args = parser.parse_args() api_key = get_api_key(args) if not api_key: print("Error: API key is not set. Use --api-key or set GEMINI_API_KEY environment variable.", file=sys.stderr) sys.exit(1) # 1. Handle Batch JSON execution if args.batch: if not os.path.exists(args.batch): print(f"Error: Batch JSON file '{args.batch}' not found.", file=sys.stderr) sys.exit(1) try: with safe_user_path(args.batch).open("r", encoding="utf-8") as f: jobs = json.load(f) if not isinstance(jobs, list): print("Error: Batch JSON file must contain a list/array of job objects.", file=sys.stderr) sys.exit(1) except Exception as e: print(f"Error parsing Batch JSON: {e}", file=sys.stderr) sys.exit(1) print(f"Loaded {len(jobs)} jobs from batch JSON. Running with concurrency={args.concurrency}...") # 2. Handle Prompts File execution elif args.prompts_file: if not os.path.exists(args.prompts_file): print(f"Error: Prompts file '{args.prompts_file}' not found.", file=sys.stderr) sys.exit(1) jobs = [] with safe_user_path(args.prompts_file).open("r", encoding="utf-8") as f: for line in f: line = line.strip() if line and not line.startswith("#"): jobs.append({ "prompt": line, "aspect_ratio": args.aspect_ratio, "duration": args.duration, "image": args.image, "video": args.video, "model": args.model, "strip_audio": args.strip_audio, "previous_interaction_id": args.previous_interaction_id }) print(f"Loaded {len(jobs)} prompts from text file. Running with concurrency={args.concurrency}...") # 3. Handle standard single prompt execution else: if not args.prompt: parser.print_help() sys.exit(1) output_path = args.output if args.output else "media/output.mp4" try: generate_video( prompt=args.prompt, api_key=api_key, model=args.model, aspect_ratio=args.aspect_ratio, duration=args.duration, image_path=args.image, video_path=args.video, output_path=output_path, strip_audio=args.strip_audio, previous_interaction_id=args.previous_interaction_id ) sys.exit(0) except Exception as e: print(f"Error: Generation failed: {e}", file=sys.stderr) sys.exit(1) # Parallel Execution Loop if not jobs: print("Warning: No valid jobs found to execute.") sys.exit(0) results = [] with ThreadPoolExecutor(max_workers=args.concurrency) as executor: futures = {executor.submit(run_job, job, api_key): job for job in jobs} for future in as_completed(futures): results.append(future.result()) # Print Batch Results Summary print("\n" + "="*50) print("BATCH PARALLEL EXECUTION SUMMARY") print("="*50) success_count = sum(1 for r in results if r["status"] == "SUCCESS") failed_count = sum(1 for r in results if r["status"] == "FAILED") skipped_count = sum(1 for r in results if r["status"] == "SKIPPED") print(f"Total: {len(results)} | Success: {success_count} | Failed: {failed_count} | Skipped: {skipped_count}\n") for r in results: status_str = r["status"] prompt = r["job"].get("prompt") if r["status"] == "SUCCESS": print(f" [{status_str}] '{prompt}' -> {r['output_path']}") else: print(f" [{status_str}] '{prompt}' -> Error: {r.get('error')}") print("="*50) if failed_count > 0: sys.exit(1) sys.exit(0) if __name__ == "__main__": main() -
inspect_video.py 6.2 KB
#!/usr/bin/env python3 import argparse import json import os import subprocess import sys def format_size(size_bytes): """Formats file size in bytes to a human-readable string.""" try: size_bytes = int(size_bytes) except (ValueError, TypeError): return "Unknown size" for unit in ['B', 'KB', 'MB', 'GB']: if size_bytes < 1024.0: return f"{size_bytes:.2f} {unit}" size_bytes /= 1024.0 return f"{size_bytes:.2f} TB" def parse_fps(fps_str): """Parses fractional frame rates like '30/1' or '24000/1001' into floats.""" if not fps_str: return "Unknown" if "/" in fps_str: try: num, den = map(float, fps_str.split("/")) if den != 0: val = num / den if val.is_integer(): return f"{int(val)} fps" return f"{val:.2f} fps" except (ValueError, ZeroDivisionError): pass try: val = float(fps_str) if val.is_integer(): return f"{int(val)} fps" return f"{val:.2f} fps" except ValueError: return fps_str def inspect_video(file_path, raw=False): """Runs ffprobe on the video file and returns parsed metadata dictionary.""" if not os.path.exists(file_path): raise FileNotFoundError(f"File not found: {file_path}") # Check if ffprobe is available try: subprocess.run(["ffprobe", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) except (subprocess.SubprocessError, FileNotFoundError): raise RuntimeError("ffprobe is not installed or not found in system PATH.") cmd = [ "ffprobe", "-v", "error", "-show_format", "-show_streams", "-of", "json", file_path ] result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=True) data = json.loads(result.stdout) if raw: return data # Extract format level details fmt = data.get("format", {}) duration = fmt.get("duration") size_bytes = fmt.get("size") bitrate = fmt.get("bit_rate") # Format files size size_str = format_size(size_bytes) if size_bytes else "Unknown" # Parse duration try: duration_val = float(duration) if duration else 0.0 duration_str = f"{duration_val:.2f}s" except ValueError: duration_str = "Unknown" duration_val = None # Parse bitrate try: bitrate_kbps = f"{int(float(bitrate) / 1000)} kbps" if bitrate else "Unknown" except ValueError: bitrate_kbps = "Unknown" video_streams = [s for s in data.get("streams", []) if s.get("codec_type") == "video"] audio_streams = [s for s in data.get("streams", []) if s.get("codec_type") == "audio"] has_video = len(video_streams) > 0 has_audio = len(audio_streams) > 0 video_info = {} if has_video: v = video_streams[0] width = v.get("width") height = v.get("height") codec = v.get("codec_name", "Unknown").upper() r_fps = parse_fps(v.get("r_frame_rate")) avg_fps = parse_fps(v.get("avg_frame_rate")) # Prefer r_frame_rate but fallback to avg fps = r_fps if r_fps != "0 fps" and r_fps != "Unknown" else avg_fps video_info = { "resolution": f"{width}x{height}" if width and height else "Unknown", "width": width, "height": height, "fps": fps, "codec": codec, "duration": v.get("duration") } audio_info = {} if has_audio: a = audio_streams[0] codec = a.get("codec_name", "Unknown").upper() channels = a.get("channels", "Unknown") sample_rate = a.get("sample_rate") sample_rate_khz = f"{float(sample_rate)/1000:.1f} kHz" if sample_rate else "Unknown" audio_info = { "codec": codec, "channels": channels, "sample_rate": sample_rate_khz } return { "file_name": os.path.basename(file_path), "file_size": size_str, "size_bytes": size_bytes, "duration": duration_str, "duration_seconds": duration_val, "bitrate": bitrate_kbps, "has_video": has_video, "video": video_info, "has_audio": has_audio, "audio": audio_info } def print_terminal_report(info): """Prints an aligned terminal report.""" print(f"\nVideo Inspection Report: {info['file_name']}") print("=" * 50) print(f"File Size : {info['file_size']}") print(f"Duration : {info['duration']}") print(f"Bitrate : {info['bitrate']}") print("\nVideo Stream Details:") if info["has_video"]: v = info["video"] print(f" * Resolution : {v['resolution']}") print(f" * Frame Rate : {v['fps']}") print(f" * Codec : {v['codec']}") else: print(" * No Video Stream Found.") print("\nAudio Stream Details:") if info["has_audio"]: a = info["audio"] print(" * Status : Audio Present") print(f" * Codec : {a['codec']}") print(f" * Channels : {a['channels']}") print(f" * Sample Rate: {a['sample_rate']}") else: print(" * Status : No Audio Stream Present") print() def main(): parser = argparse.ArgumentParser(description="Inspect video details (duration, frame rate, resolution, audio presence) using ffprobe.") parser.add_argument("file", help="Path to the video file to inspect") parser.add_argument("--json", action="store_true", help="Output parsed summary in JSON format") parser.add_argument("--raw", action="store_true", help="Output raw unmodified ffprobe JSON data") args = parser.parse_args() try: if args.raw: info = inspect_video(args.file, raw=True) print(json.dumps(info, indent=2)) else: info = inspect_video(args.file, raw=False) if args.json: print(json.dumps(info, indent=2)) else: print_terminal_report(info) except Exception as e: print(f"Error inspecting video: {e}", file=sys.stderr) sys.exit(1) if __name__ == "__main__": main() -
prep_video.py 10.5 KB
#!/usr/bin/env python3 import argparse import os import subprocess import sys from inspect_video import inspect_video, format_size def parse_timecode(time_str, total_duration=None): """Parses a time string (seconds, MM:SS, HH:MM:SS, or 'last') into float seconds.""" if not time_str: return 0.0 time_str = time_str.strip().lower() if time_str == "last": if total_duration is None: raise ValueError("Total duration is required to calculate 'last' starting point.") target_dur = 10.0 if total_duration <= target_dur: return 0.0 return total_duration - target_dur if ":" in time_str: parts = time_str.split(":") if len(parts) == 2: # MM:SS m, s = map(float, parts) return m * 60.0 + s elif len(parts) == 3: # HH:MM:SS h, m, s = map(float, parts) return h * 3600.0 + m * 60.0 + s else: raise ValueError(f"Invalid timecode format: '{time_str}'. Use HH:MM:SS or MM:SS.") try: return float(time_str) except ValueError: raise ValueError(f"Invalid timecode: '{time_str}'. Must be float seconds, HH:MM:SS, or 'last'.") def prep_video(input_path, output_path, start_time_str=None, duration=10, fps=None, resolution=None, strip_audio=False): """Preps a video file by trimming, optionally re-encoding to target fps and resolution.""" if not os.path.exists(input_path): raise FileNotFoundError(f"Input file not found: {input_path}") # Check if ffmpeg is available try: subprocess.run(["ffmpeg", "-version"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) except (subprocess.SubprocessError, FileNotFoundError): raise RuntimeError("ffmpeg is not installed or not found in system PATH.") # Inspect input video first print(f"Analyzing source video: {os.path.basename(input_path)}...") source_info = inspect_video(input_path) total_duration = source_info.get("duration_seconds", 0.0) # Resolve start time if start_time_str is None: if total_duration and total_duration > 10.0 and sys.stdin.isatty(): print(f"\nThe input video is longer than 10s ({total_duration:.2f}s).") print("Please choose a 10s segment to trim:") print(" 1) First 10 seconds [default]") print(" 2) Last 10 seconds") print(" 3) Custom starting timecode (e.g., MM:SS, HH:MM:SS, or seconds)") try: choice = input("Your choice [1/2/3, default 1]: ").strip() if choice == "2": start_time_str = "last" elif choice == "3": custom_start = input("Enter starting timecode (e.g., 00:03 or 15): ").strip() start_time_str = custom_start if custom_start else "0" else: start_time_str = "0" except (KeyboardInterrupt, EOFError): print("\nNo input received. Defaulting to first 10 seconds.") start_time_str = "0" else: start_time_str = "0" try: start_seconds = parse_timecode(start_time_str, total_duration) except Exception as e: raise ValueError(f"Timecode parsing failed: {e}") if start_seconds < 0 or (total_duration and start_seconds >= total_duration): raise ValueError(f"Start time {start_seconds}s is out of bounds for video of length {total_duration}s.") # Construct output path if not specified if not output_path: os.makedirs("media", exist_ok=True) base_name = os.path.basename(input_path) name, ext = os.path.splitext(base_name) output_path = os.path.join("media", f"prepped_{name}.mp4") # Check if the source video file is large (>25MB) size_bytes_str = source_info.get("size_bytes") is_large = False try: if size_bytes_str and int(size_bytes_str) > 25 * 1024 * 1024: is_large = True except (ValueError, TypeError): pass # Target resolution parsing scale_filter = None orig_width = None orig_height = None if "video" in source_info: try: orig_width = int(source_info["video"].get("width")) orig_height = int(source_info["video"].get("height")) except (ValueError, TypeError): pass if resolution: try: target_w, target_h = map(int, resolution.lower().split("x")) if orig_width and orig_height: # Scale to fit target_w and target_h while preserving aspect ratio scale_factor = min(target_w / orig_width, target_h / orig_height) width = int(orig_width * scale_factor) height = int(orig_height * scale_factor) else: width, height = target_w, target_h # Ensure divisible by 2 for standard decoders/encoders width = (width // 2) * 2 height = (height // 2) * 2 scale_filter = f"scale={width}:{height}" resolution = f"{width}x{height}" except ValueError: raise ValueError(f"Invalid resolution: '{resolution}'. Format must be WIDTHxHEIGHT (e.g. 1280x720).") elif is_large: if orig_width and orig_height: # Scale down large videos proportionally (max 1280x720 for landscape, 720x1280 for portrait) if orig_width >= orig_height: max_w, max_h = 1280, 720 else: max_w, max_h = 720, 1280 scale_factor = min(max_w / orig_width, max_h / orig_height) if scale_factor < 1.0: width = int(orig_width * scale_factor) height = int(orig_height * scale_factor) else: width, height = orig_width, orig_height else: width, height = 1280, 720 # Ensure divisible by 2 width = (width // 2) * 2 height = (height // 2) * 2 resolution = f"{width}x{height}" print(f"\nRecommendation: Source video is very large ({source_info.get('file_size')}).") print(" Automatically scaling to optimize upload times for Gemini Omni Flash.") scale_filter = f"scale={width}:{height}" fps_spec = f"{fps} fps" if fps else "Original frame rate" print(f"\nPreparing Video Processing:") print(f" * Source Duration: {total_duration:.2f}s") print(f" * Trim Range : Start at {start_seconds:.2f}s | Length {duration:.2f}s") if resolution: print(f" * Encoding Specs : {width}x{height} @ {fps_spec}") else: print(f" * Encoding Specs : Original Resolution @ {fps_spec}") print(f" * Target Path : {output_path}") print("=" * 50) # ffmpeg command construction cmd = [ "ffmpeg", "-y", # Overwrite output "-ss", str(start_seconds), # Seek start "-i", input_path, # Input file "-t", str(duration), # Duration to copy ] if scale_filter: cmd.extend(["-vf", scale_filter]) cmd.extend([ "-c:v", "libx264", # Standard H264 video codec "-pix_fmt", "yuv420p", # Standard pixel format for web/Gemini compatibility ]) if fps: cmd.extend(["-r", str(fps)]) # Output frame rate if requested if strip_audio or not source_info.get("has_audio", False): if not source_info.get("has_audio", False) and not strip_audio: print("No audio stream detected in source video. Disabling audio output.") else: print("Stripping audio stream from video as requested.") cmd.append("-an") # Disable audio streams completely else: cmd.extend([ "-c:a", "aac", # Convert audio to standard AAC "-b:a", "128k", # Standard audio bitrate "-ac", "2", # Convert to stereo ]) cmd.append(output_path) print("Running ffmpeg encoding...") process = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) if process.returncode != 0: print("Error: ffmpeg failed. Stderr output follows:", file=sys.stderr) print(process.stderr, file=sys.stderr) raise RuntimeError("ffmpeg execution failed.") print("Video preparation completed successfully!") print("=" * 50) # Call inspection tool on output to print clean specs output_info = inspect_video(output_path) return output_info def main(): parser = argparse.ArgumentParser(description="Prep videos for editing (trimming, re-encoding to target fps and resolution).") parser.add_argument("file", help="Path to the source video file to prep") parser.add_argument("--start", "-s", default=None, help="Start timecode (seconds, MM:SS, HH:MM:SS, or 'last' for last 10s). Default: 0 (or prompted if > 10s)") parser.add_argument("--duration", "-d", type=int, default=10, help="Duration of trimmed segment in seconds. Default: 10") parser.add_argument("--fps", "-r", type=int, default=None, help="Target frame rate. Default: None (keep original frame rate)") parser.add_argument("--resolution", "-g", default=None, help="Target resolution (e.g., 1280x720). Default: None (keep original resolution)") parser.add_argument("--output", "-o", help="Custom output path. Defaults to media/prepped_<original_name>.mp4") parser.add_argument("--strip-audio", "-a", action="store_true", help="Completely strip/disable audio stream so the model can generate new audio") args = parser.parse_args() try: info = prep_video( input_path=args.file, output_path=args.output, start_time_str=args.start, duration=args.duration, fps=args.fps, resolution=args.resolution, strip_audio=args.strip_audio ) # Display final output report print(f"\nPrepped Video Specifications: {info['file_name']}") print("=" * 50) print(f"File Size : {info['file_size']}") print(f"Duration : {info['duration']}") print(f"Bitrate : {info['bitrate']}") print(f"Resolution : {info['video']['resolution']}") print(f"Frame Rate : {info['video']['fps']}") print(f"Video Codec : {info['video']['codec']}") if info["has_audio"]: print(f"Audio Spec : {info['audio']['codec']} | {info['audio']['channels']} ch | {info['audio']['sample_rate']}") print() except Exception as e: print(f"Error prepping video: {e}", file=sys.stderr) sys.exit(1) if __name__ == "__main__": main()
-
-
upload_file.py 9.2 KB
#!/usr/bin/env python3 """ Uploads a file to the Gemini Files API and waits for it to become ACTIVE. Uses the official google-genai SDK. """ import argparse import json import mimetypes import os import sys import time from google import genai from google.genai import types def get_api_key(args): """Retrieves API key from command args or environment.""" if args.api_key: return args.api_key return os.environ.get("GEMINI_API_KEY") def detect_mime_type(file_path): """Determines MIME type based on file extension, falling back to standard mimetypes module.""" ext = os.path.splitext(file_path)[1].lower() mime_map = { ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".webp": "image/webp", ".mp4": "video/mp4", ".mp3": "audio/mpeg", ".wav": "audio/wav", ".pdf": "application/pdf", ".txt": "text/plain", } if ext in mime_map: return mime_map[ext] mime_type, _ = mimetypes.guess_type(file_path) if mime_type: return mime_type return "application/octet-stream" def upload_file(file_path, display_name=None, api_key=None): """Performs an upload using google-genai SDK, with automatic pre-processing for large videos.""" file_size = os.path.getsize(file_path) mime_type = detect_mime_type(file_path) # Large video file size check (>25MB) is_video = mime_type.startswith("video/") if is_video and file_size > 25 * 1024 * 1024: size_mb = file_size / (1024 * 1024) print(f"\nWARNING: Video file '{file_path}' is very large ({size_mb:.2f} MB)!") print("Note: Gemini Omni Flash is optimized for 10s videos at 720p and 24fps. Uploading very large or") print("high-resolution videos will significantly increase upload times and may cause Out-Of-Memory (OOM) errors.") # Determine if terminal is interactive if sys.stdin.isatty(): print("\nWould you like to automatically pre-process this video first using prep_video.py?") print("This will trim, scale, and optimize the video to ensure a fast, OOM-safe upload.") try: choice = input("Pre-process video? [Y/n]: ").strip().lower() if choice in ("", "y", "yes"): prepped_output_path = os.path.join("media", f"prepped_{os.path.basename(file_path)}") os.makedirs("media", exist_ok=True) # Resolve prep_video.py script path import subprocess prep_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "video", "prep_video.py") if not os.path.exists(prep_script): prep_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "prep_video.py") cmd = [sys.executable, prep_script, file_path, "--output", prepped_output_path] print(f"Running: {' '.join(cmd)}") try: result = subprocess.run(cmd) if result.returncode == 0 and os.path.exists(prepped_output_path): file_path = prepped_output_path file_size = os.path.getsize(file_path) print(f"\nPre-processing completed successfully! Proceeding with upload of prepped video ({file_size / (1024*1024):.2f} MB)...") else: raise RuntimeError("Error: Video pre-processing failed. Proceeding with original file upload is not recommended.") except Exception as e: raise RuntimeError(f"Error executing prep_video.py: {e}") else: proceed_choice = input("Do you want to proceed with uploading the original large video anyway? [y/N]: ").strip().lower() if proceed_choice not in ("y", "yes"): raise RuntimeError("Upload cancelled by user. Please pre-process the video manually first.") except (KeyboardInterrupt, EOFError): raise RuntimeError("\nNo input received. Upload cancelled to prevent OOM.") else: # Non-interactive mode if file_size > 100 * 1024 * 1024: # Block files larger than 100MB in non-interactive mode err_msg = ( f"Error: Video file is extremely large ({size_mb:.2f} MB) and script is running in non-interactive mode.\n" "To prevent Out-Of-Memory (OOM) errors, upload has been blocked.\n" "Please pre-process the video first using prep_video.py." ) raise RuntimeError(err_msg) else: print("Proceeding with upload in non-interactive mode...", file=sys.stderr) if not display_name: display_name = os.path.basename(file_path) print(f"Preparing upload of '{file_path}' ({file_size} bytes, type: {mime_type})...") # Step 1: Initialize Client client = genai.Client(api_key=api_key) # Step 2: Upload file using SDK print("Uploading file bytes using google-genai SDK...") try: config = types.UploadFileConfig( display_name=display_name, mime_type=mime_type, ) file_obj = client.files.upload(file=file_path, config=config) # Convert Pydantic File model to dictionary with both camelCase and snake_case keys for compatibility file_dict = json.loads(file_obj.model_dump_json()) # Add camelCase field for mimeType if "mime_type" in file_dict: file_dict["mimeType"] = file_dict["mime_type"] return file_dict except Exception as e: raise RuntimeError(f"Error uploading file via SDK: {e}") def wait_for_active(file_name, api_key, poll_interval=3, max_attempts=30, backoff_factor=1.5, max_interval=30): """Polls the file status until state is ACTIVE or FAILED using exponential backoff via the SDK.""" print(f"Waiting for file {file_name} to finish processing...") client = genai.Client(api_key=api_key) attempt = 0 current_interval = poll_interval consecutive_errors = 0 max_consecutive_errors = 5 while attempt < max_attempts: try: file_obj = client.files.get(name=file_name) state = file_obj.state # Extract state name as a string (handling both Enum and String) state_str = state.name if hasattr(state, "name") else str(state) # Reset consecutive errors on successful API response consecutive_errors = 0 if state_str == "ACTIVE": print("File is ACTIVE and ready for generations!") file_dict = json.loads(file_obj.model_dump_json()) if "mime_type" in file_dict: file_dict["mimeType"] = file_dict["mime_type"] return file_dict elif state_str == "FAILED": raise RuntimeError("Error: File processing failed on the backend.") print(f"Current state: {state_str}. Retrying in {current_interval:.1f}s...") time.sleep(current_interval) # Increase interval for the next poll (backoff) current_interval = min(current_interval * backoff_factor, max_interval) attempt += 1 except Exception as e: consecutive_errors += 1 if consecutive_errors >= max_consecutive_errors: raise RuntimeError(f"Error: Too many consecutive errors checking status ({e}). Exiting.") print(f"Warning: Error checking status ({e}). Retrying in {current_interval:.1f}s...") time.sleep(current_interval) current_interval = min(current_interval * backoff_factor, max_interval) attempt += 1 raise RuntimeError(f"Error: Maximum polling attempts ({max_attempts}) reached. File is still not ACTIVE.") def main(): parser = argparse.ArgumentParser(description="Upload files to Gemini Files API using google-genai SDK.") parser.add_argument("file", help="Path to the file to upload") parser.add_argument("--name", help="Custom display name for the file") parser.add_argument("--api-key", help="Gemini API Key (overrides env)") parser.add_argument("--no-wait", action="store_true", help="Don't wait for ACTIVE status") args = parser.parse_args() api_key = get_api_key(args) if not api_key: print("Error: API key is not set. Use --api-key or set GEMINI_API_KEY environment variable.", file=sys.stderr) sys.exit(1) if not os.path.exists(args.file): print(f"Error: File '{args.file}' not found.", file=sys.stderr) sys.exit(1) try: file_meta = upload_file(args.file, args.name, api_key) file_name = file_meta.get("name") print(f"File metadata created:") print(f" Name: {file_name}") print(f" URI: {file_meta.get('uri')}") print(f" Type: {file_meta.get('mimeType')}") if not args.no_wait: file_meta = wait_for_active(file_name, api_key) print("\nFile upload successfully completed! JSON Output:") print(json.dumps(file_meta, indent=2)) sys.exit(0) except Exception as e: print(f"Error: {e}", file=sys.stderr) sys.exit(1) if __name__ == "__main__": main()
-
-
SKILL.md 16.5 KB
--- name: gemini-omni-flash-api description: Use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,... risk: critical source: https://github.com/google-gemini/gemini-skills/tree/main/skills/gemini-omni-flash-api source_repo: google-gemini/gemini-skills source_type: official date_added: 2026-07-01 license: Apache-2.0 license_source: https://github.com/google-gemini/gemini-skills/blob/main/LICENSE --- # Gemini Omni Flash Skill ## When to Use Use this skill when you need use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg,... This skill uses the Gemini Omni Flash model (`gemini-omni-flash-preview`) to perform text to video generation, image to video generation and video editing. > [!WARNING] > **Important Regional Restrictions**: Uploading videos to use for video edits is **NOT** available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (`total_output_tokens: 0` or no video content), it is likely due to this restriction. ## Core capabilities 1. **Video editing and refinement**: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting. 2. **Text to video**: Generating videos from a text prompt. 3. **First-frame to video**: Generating videos from a single input image. 4. **Image-referenced generation**: Using style, character, or object references from images to guide video generation. ## Workflow 1. **Analyze request**: Determine the target task (e.g., first-frame-to-video, reference-guided editing) and identify any input media assets. 2. **Run SDK scripts**: * Directly run the appropriate utility (`scripts/video/generate_video.py` or `scripts/upload_file.py`). * Configure settings like `--aspect-ratio` (e.g. `16:9`, `9:16`) and `--duration` (any integer between `3` and `10` seconds, e.g. `3`, `5`, `10`). 3. **Retrieve and process output**: Outputs are saved to the local filesystem (e.g. `media/`). Report back the completed media path to the user. ## Reference Documentation * **Interactions API**: All operations and state management for the Gemini Omni Flash model (`gemini-omni-flash-preview`) are handled via the [Interactions API](https://ai.google.dev/gemini-api/docs/interactions-overview). * **Files API**: Input media files (such as reference images and videos) must be uploaded via the [Files API](https://ai.google.dev/gemini-api/docs/interactions/files) first before being referenced in generations. The uploaded file URI and MIME type are then included in the `interactions.create` input parts array. * **[Interactions API Skill Reference](https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-interactions-api/SKILL.md)**: Platform-wide guidelines, current model specifications, and SDK usage rules for the Interactions API. ## Dependencies and Prerequisites * **Python SDK (`google-genai`)**: Requires `google-genai >= 2.10.0` (Python) to support the new `interactions` client attribute. Install or upgrade using: ```bash pip install -U google-genai ``` * **Python Runtime**: Requires **Python >= 3.10** (for compatibility with modern `google-genai` SDK types and methods). * **ffmpeg & ffprobe**: `prep_video.py`, `inspect_video.py`, and `generate_video.py` (when stripping audio via `--strip-audio`) require `ffmpeg` and `ffprobe` binaries installed and available in your system `PATH`. ## Available scripts Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API. 1. **[upload_file.py](scripts/upload_file.py)**: Uploads local media (images and videos) to the Files API and polls until `ACTIVE`. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing with `prep_video.py` first to speed up the upload. ```bash ./scripts/upload_file.py path/to/image.png ``` 2. **[generate_video.py](scripts/video/generate_video.py)**: Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload. * **Text to video**: ```bash ./scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4 ``` * **Image to video (first frame and reference)**: ```bash ./scripts/video/generate_video.py "The waves crash against the shore." --image reference.png --output media/waves.mp4 ``` * **Video interpolation**: Provide exactly two images as keyframes to generate a transition video between them: ```bash ./scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --image start.png --image end.png --output media/interpolation.mp4 ``` * **Video editing (keep original audio)**: ```bash ./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --output media/anime_style.mp4 ``` * **Video editing (regenerate all audio from scratch)**: ```bash ./scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --strip-audio --output media/anime_style_new_audio.mp4 ``` * **Turn-by-turn video editing (edit previous interaction)**: Edit a prior video generation without re-uploading assets by passing the interaction ID: ```bash ./scripts/video/generate_video.py "Change the setting to a snowy winter wonderland." --previous-interaction-id "abc123xyz..." --output media/winter_wonderland.mp4 ``` * **Parallel batch execution (prompts file)**: Run multiple prompts from a line-by-line text file concurrently: ```bash ./scripts/video/generate_video.py --prompts-file prompts.txt --concurrency 3 ``` * **Parallel batch execution (JSON config)**: Execute fully configured, distinct generation and editing jobs in parallel: ```bash ./scripts/video/generate_video.py --batch jobs.json --concurrency 3 ``` *Example `jobs.json`:* ```json [ { "prompt": "Transform the style to Japanese anime.", "video": "input.mp4", "output": "media/anime_style.mp4", "strip_audio": false, "aspect_ratio": "16:9" }, { "prompt": "A smooth timelapse from sunrise to sunset.", "image": ["start.png", "end.png"], "output": "media/interpolation.mp4" } ] ``` 3. **[inspect_video.py](scripts/video/inspect_video.py)**: Inspects a local video file (using `ffprobe`) to check its duration, resolution, frame rate (FPS), audio stream presence, and format details. ```bash ./scripts/video/inspect_video.py media/output.mp4 ``` * To get a pre-parsed, structured JSON summary: ```bash ./scripts/video/inspect_video.py media/output.mp4 --json ``` * To get the complete, unmodified `ffprobe` raw JSON dump: ```bash ./scripts/video/inspect_video.py media/output.mp4 --raw ``` 4. **[prep_video.py](scripts/video/prep_video.py)**: Normalizes, trims, and formats any video file to fit standard Gemini Omni Flash generation and editing limits. It handles timecode-based trimming, optional frame rate conversion, and proportional scaling of large videos (max 1280x720 for landscape, 720x1280 for portrait) to optimize upload times without stretching. If the video is longer than 10 seconds and the script is run interactively (in a TTY), it prompts the user to select the first 10s, last 10s, or enter a custom timecode (defaulting to the first 10s). * **Trim first 10s (default)**: ```bash ./scripts/video/prep_video.py path/to/source.mp4 ``` or explicitly specify the start and duration: ```bash ./scripts/video/prep_video.py path/to/source.mp4 --start 0 --duration 10 ``` * **Trim last 10s** (automatically calculates starting point based on source length): ```bash ./scripts/video/prep_video.py path/to/source.mp4 --start last ``` * **Trim 10s starting at specific timecode** (MM:SS or HH:MM:SS): ```bash ./scripts/video/prep_video.py path/to/source.mp4 --start 00:03 --output media/custom.mp4 ``` * **Custom frame rate and resolution**: ```bash ./scripts/video/prep_video.py path/to/source.mp4 --fps 30 --resolution 1920x1080 ``` * **Strip audio for audio regeneration**: ```bash ./scripts/video/prep_video.py path/to/source.mp4 --strip-audio --output media/video_with_no_audio.mp4 ``` ## Using tags in prompts to set image roles You can use tags in your prompt to make it clear whether each uploaded media is an initial frame or a reference. ### 1. Simple tags (recommended) For simple cases where image roles are clear from the prompt, you can bind images to roles directly: * **`<FIRST_FRAME>`**: Use the image as the starting frame of the video, for example: `<FIRST_FRAME> a woman is walking` * **`<IMAGE_REF_N>`**: Use the image as a reference, for example: `in the style of <IMAGE_REF_0> a woman <IMAGE_REF_1> is walking` (combines style reference from the first image and subject reference from the second image). Image references start from 0. An example with 6 reference images: ```none [0-3s] A studio fashion sequence. Starting with woman <IMAGE_REF_0>, she is holding <IMAGE_REF_1> [3-6s] Then we see the man <IMAGE_REF_2> holding <IMAGE_REF_3> [6-10s] And finally another woman <IMAGE_REF_4> who is holding <IMAGE_REF_5> while walking. ``` ### 2. Explicitly declare sources and references For more complex cases with multiple images and multiple roles, you can use explicit prefix tags paired with natural language instruction suffixes. * **Declaring sources and reference images**: * `[# Sources <FIRST_FRAME>@Image1]` will use the first image as the starting frame. * `[# References <IMAGE_REF_0>@Image1]` will use the first image as a reference. * `[# References <IMAGE_REF_1>@Image2]` will use the second image as a reference. * `[# References <IMAGE_REF_0>@Image1 <IMAGE_REF_1>@Image2]` will use both images as references. * `[# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2]` will use the first image as the starting frame and the second image as a reference. * **Guiding instructions**: Add guiding instructions at the end of your prompt: * For starting frame: `"Use the given image as the starting frame."` * For reference images: `"Use the given image(s) as references for video generation. The images should not be used as literal initial frames."` * *Example Expanded Prompt*: ```none [# Sources <FIRST_FRAME>@Image1] [# References <IMAGE_REF_0>@Image2] a woman <IMAGE_REF_0> is walking. Use Image1 as the starting frame. Use Image2 as a reference for the video generation. ``` ## Audio handling in video editing When editing a source video that contains audio, you must choose between keeping the original audio or regenerating all audio from scratch. * **Keep original audio**: By default, Gemini Omni Flash preserves the existing audio layer (though it may modify or adapt it slightly during generation). Use this when the original background music, dialogue, or sound effects are desired. * **Regenerate all audio from scratch**: If you want Gemini Omni Flash to re-create a brand-new audio layer tailored to the new visual style or prompt, you **must** upload the video with its audio stream stripped out. If any audio stream is present, Gemini Omni Flash will attempt to preserve/modify it instead of starting from scratch. * Use `--strip-audio` (or `-a`) when pre-processing with `scripts/video/prep_video.py` or executing `scripts/video/generate_video.py`. * This forces Gemini Omni Flash to perform full audio generation. ## Prompting Gemini Omni Flash ### Single scene By default Gemini Omni Flash will try to create a video with a few different shots. It'll attempt to craft an interesting narrative based on the prompt. If you need the output video to contain a single scene, you must prompt for that: * In a single unbroken scene * In a single continuous shot * No scene cuts For example: ```none Continuous, unbroken handheld shot of a fluffy tabby cat sitting on a sunny windowsill, looking out into a leafy garden. The cat's tail twitches slowly, and its ears rotate slightly toward ambient noises. Sunbeams illuminate dust motes in the air. Sound design: Gentle breeze, distant bird chirps, quiet mechanical purring. No dialogue. ``` ### Removing unwanted elements If generations contain things you don't want, you can include simple negatives to avoid them: * No dialogue * No embellishments * No extra sound effects ### Prompts for editing Simple prompts work best for editing. Overly descriptive prompts can lead to unintended changes. For example: * Make this video anime * Make the phone invisible * Put a fashionable hat on this person * Change the lighting to be more dramatic * Change the text on the sign to say "Gemini Omni Flash" * Add a cat that jumps onto his lap, he begins to pet it When editing a specific aspect of the video, it can help to include: "Keep everything else the same". ### Prompting the audio By default the model will try to generate an appropriate audio track for a video. This might not always be what you want. You can use your prompt to describe the type of audio you want. This is especially important if you want music in your video: * Include calm background music * The video has a high energy techno beat * The audio is a low tinny radio broadcast in the background, playing a song * Audio design: [a description of the audio you want] ### When things should happen You can prompt for things to happen at specific times in the video, there is no precise syntax needed and you can use natural language. This is especially useful in creating your own scene cuts, rhythm or rapid fire sequences. Simple examples: * after 3 seconds, a woman enters the scene * at 5s the chorus starts in the background audio * every 2s cut to a new frame * in a rapid fire sequence, every half a second (12 frames at 24fps) change the scene to a new location You can also use a timecode syntax: ```none [0-3s] A person is walking [3-6s] They stop and turn around [6-10s] They start running ``` ### Meta prompting Rather than specifying everything directly in a prompt, you can ask the model to pay attention to certain things. You can give Gemini Omni Flash these sorts of prompts verbatim: * Consider micro-detail, expression and timing to create a very rich, detailed but entirely natural scene. * Be extremely detailed in your descriptions of characters and environments. Apply costume design principles to characters. Be very specific about the people, items and objects in the scene. * Include plenty of appropriate detail in the background elements to make the scene feel realistic and natural. * Make a rapid fire video that shows a different rare [thing] every 1s, upbeat music, include text to label the thing. ### Text in videos works really well Unlike previous video models, text in Gemini Omni Flash videos works really well. You can include decent amounts of text in your video and it will be rendered in a way that is correct and readable. If there will be naturally occurring text in your video, even in background elements, it can help to define what it should say. For example: * One word on the screen at a time: "did, you, know, that, Omni, can, do, awesome, text?" Each word appears for 1s with a different animated style. No dialogue. * There is a street sign that says: "This is an AI generation by Omni", there is a storefront that says: "All you need AI", there's a car with the number plate: "OMN111" ## Limitations - Use this skill only when the task clearly matches its upstream product or API scope. - Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes. - Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.