Claude Skill

ai-infrastructure-replicate

Replicate SDK patterns for TypeScript/Node.js -- client setup, predictions, streaming, webhooks, file handling, model versioning, deployments, and training

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download agents-inc-skills-dist_plugins_ai-infrastructure-replicate_skills_ai-infrastructure-replicate-3a51ef5.zip · 17 KB
Part of agents-inc/skills — 130 skills

Install

skills CLI npx skills add https://github.com/agents-inc/skills/tree/main/dist/plugins/ai-infrastructure-replicate/skills/ai-infrastructure-replicate
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install agents-inc-skills@llmmart
Git git clone https://github.com/agents-inc/skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole agents-inc/skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Replicate SDK Patterns

Quick Guide: Use the replicate npm package to run open-source ML models on serverless GPUs. Use replicate.run() for synchronous execution that returns output directly, replicate.stream() for SSE-based streaming, or replicate.predictions.create() for async background jobs with webhook notifications. Models are referenced as owner/model (uses latest version) or owner/model:version (pinned). File outputs are FileOutput objects implementing ReadableStream. Cold starts are expected for infrequently-used models -- use deployments with min_instances to keep models warm.


<critical_requirements>

CRITICAL: Before Using This Skill

All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering, import type, named constants)

(You MUST never hardcode API tokens -- always use environment variables via process.env.REPLICATE_API_TOKEN)

(You MUST handle FileOutput objects for models that return files -- do not assume outputs are plain strings or URLs)

(You MUST validate webhooks using validateWebhook() from the replicate package -- never trust unverified webhook payloads)

(You MUST account for cold starts when running infrequently-used models -- use deployments for latency-sensitive applications)

(You MUST specify model versions (owner/model:version) in production to ensure reproducible results -- unversioned references use the latest, which can change)

</critical_requirements>


Auto-detection: Replicate, replicate, replicate.run, replicate.stream, replicate.predictions, replicate.deployments, replicate.trainings, replicate.models, FileOutput, validateWebhook, REPLICATE_API_TOKEN, serverless GPU, cold start, webhook_events_filter

When to use:

  • Running open-source ML models (Llama, Stable Diffusion, Whisper, etc.) without managing GPU infrastructure
  • Generating images, transcribing audio, running LLMs, or any ML inference via API
  • Streaming LLM output in real-time with server-sent events
  • Processing predictions asynchronously with webhook notifications
  • Fine-tuning models with custom training data
  • Running models on dedicated hardware with custom scaling via deployments

Key patterns covered:

  • Client initialization and configuration (auth, user agent, file encoding)
  • Running predictions (replicate.run(), replicate.predictions.create(), replicate.wait())
  • Streaming output (replicate.stream() with SSE events)
  • Model versioning (owner/model vs owner/model:version)
  • File input/output handling (FileOutput, file uploads, Buffer inputs)
  • Webhooks (setup, event filtering, signature validation)
  • Deployments (custom hardware, scaling, keeping models warm)
  • Training / fine-tuning

When NOT to use:

  • You need a unified multi-provider LLM SDK (OpenAI, Anthropic, Google) -- use a provider-agnostic SDK
  • You want to run models locally -- Replicate is a cloud-only serverless platform
  • You need sub-second latency guarantees without deployments -- cold starts can take minutes

Examples Index





<decision_framework>

Decision Framework

Which Execution Method to Use

Is this a user-facing LLM response?
+-- YES -> Use replicate.stream() for real-time SSE output
+-- NO -> Do you need the result immediately?
    +-- YES -> Use replicate.run() (blocks until complete)
    +-- NO -> Use replicate.predictions.create() + webhook
        +-- Need to poll instead? -> Use replicate.wait(prediction)

Model Reference Format

Are you in development/prototyping?
+-- YES -> Use owner/model (latest version, convenient)
+-- NO -> Are you in production?
    +-- YES -> Use owner/model:version_hash (pinned, reproducible)
    +-- Does the model change frequently?
        +-- YES -> Pin version, test updates explicitly
        +-- NO -> Either format works, prefer pinned

Deployments vs Direct API

Do you need consistent low latency?
+-- YES -> Create a deployment with min_instances >= 1
+-- NO -> Do you need custom hardware (A100, H100)?
    +-- YES -> Create a deployment with specific hardware
    +-- NO -> Use replicate.run() / replicate.stream() directly
        (Replicate auto-allocates hardware)

When to Use This SDK vs Other AI SDKs

Are you running open-source models on serverless GPUs?
+-- YES -> Use Replicate SDK
+-- NO -> Are you calling proprietary APIs (OpenAI, Anthropic)?
    +-- YES -> Not this skill's scope -- use provider-specific SDKs
    +-- NO -> Do you need to switch between multiple providers?
        +-- YES -> Not this skill's scope -- use a unified provider SDK
        +-- NO -> Do you want to self-host models?
            +-- YES -> Not this skill's scope -- consider Cog or vLLM
            +-- NO -> Replicate SDK is appropriate

</decision_framework>


<red_flags>

RED FLAGS

High Priority Issues:

  • Hardcoding REPLICATE_API_TOKEN in source code (security breach risk)
  • Treating FileOutput as a string (it is a ReadableStream object -- use .url() or .blob())
  • Not validating webhook signatures with validateWebhook() (allows forged webhook payloads)
  • Using replicate.run() for long-running models in request handlers (blocks the response, can timeout)

Medium Priority Issues:

  • Not pinning model versions in production (owner/model uses latest, which can change without notice)
  • Relying solely on default retry behavior for production (5 retries with exponential backoff may be too aggressive for some use cases)
  • Uploading large files as Buffer instead of hosting them at a URL (100 MiB limit on uploads)
  • Ignoring cold start latency for infrequently-used models (first request can take minutes)

Common Mistakes:

  • Confusing replicate.run() (returns output directly) with replicate.predictions.create() (returns a prediction object with status/id)
  • Destructuring image output incorrectly: const output = await replicate.run(...) instead of const [output] = await replicate.run(...) (image models return arrays)
  • Using replicate.stream() with models that do not support streaming (only language models with SSE support)
  • Forgetting that replicate.predictions.create() accepts either a version hash or a model string (owner/model) -- use version for pinned reproducibility, model for latest-version convenience
  • Not consuming the async iterator from replicate.stream() (events are lost)

Gotchas & Edge Cases:

  • Prediction inputs and outputs are automatically deleted after one hour -- persist outputs via webhooks or download immediately
  • The SDK auto-retries on 429 (rate limit) and 5xx errors -- 5 retries by default with exponential backoff. GET requests retry on 429 and 5xx; non-GET requests retry only on 429
  • replicate.stream() returns ServerSentEvent objects with .event ("output", "error", "done") and .data (string) properties
  • File uploads are limited to 100 MiB -- for larger files, host them at a URL and pass the URL as input
  • Browser usage is not supported -- the SDK requires a server-side environment (Node.js 18+, Bun, Deno, Cloudflare Workers)
  • webhook_events_filter accepts ["start", "output", "logs", "completed"] -- use ["completed"] unless you need intermediate status updates
  • The Prefer: wait header enables sync mode on the HTTP API (up to 60s), but replicate.run() already handles this automatically
  • Community models may disappear or change without warning -- pin versions and maintain fallbacks for critical workflows
  • replicate.wait() polls the API until the prediction completes -- use webhooks for production to avoid polling overhead
  • FileOutput.url() returns the underlying URL, but these URLs are temporary -- download or persist the file before it expires

</red_flags>


<critical_reminders>

CRITICAL REMINDERS

All code must follow project conventions in CLAUDE.md (kebab-case, named exports, import ordering, import type, named constants)

(You MUST never hardcode API tokens -- always use environment variables via process.env.REPLICATE_API_TOKEN)

(You MUST handle FileOutput objects for models that return files -- do not assume outputs are plain strings or URLs)

(You MUST validate webhooks using validateWebhook() from the replicate package -- never trust unverified webhook payloads)

(You MUST account for cold starts when running infrequently-used models -- use deployments for latency-sensitive applications)

(You MUST specify model versions (owner/model:version) in production to ensure reproducible results -- unversioned references use the latest, which can change)

Failure to follow these rules will produce insecure, unreliable, or unpredictable AI integrations.

</critical_reminders>

Files (skills)
  • examples
    • core.md 8.2 KB
      # Replicate SDK -- Setup, Predictions & File Handling Examples
      
      > Client initialization, production config, running predictions, file I/O, model versioning, and error handling. See [SKILL.md](../SKILL.md) for core patterns.
      
      **Related examples:**
      
      - [streaming-webhooks.md](streaming-webhooks.md) -- Streaming output, SSE events, webhooks
      - [deployments-training.md](deployments-training.md) -- Deployments, training, model management
      
      ---
      
      ## Basic Client Setup
      
      ```typescript
      // lib/replicate.ts
      import Replicate from "replicate";
      
      // Reads REPLICATE_API_TOKEN from env automatically
      const replicate = new Replicate();
      
      export { replicate };
      ```
      
      ---
      
      ## Production Configuration
      
      ```typescript
      // lib/replicate.ts
      import Replicate from "replicate";
      
      const replicate = new Replicate({
        auth: process.env.REPLICATE_API_TOKEN,
        userAgent: "my-app/1.0.0",
      });
      
      export { replicate };
      ```
      
      ### Constructor Options
      
      ```typescript
      new Replicate({
        auth: process.env.REPLICATE_API_TOKEN, // Auto-reads from env if omitted
        baseUrl: "https://api.replicate.com/v1", // Override for proxies
        userAgent: "my-app/1.0.0", // Custom user agent string
        fetch: globalThis.fetch, // Custom fetch implementation
        fileEncodingStrategy: "default", // "default" | "upload" | "data-uri"
        useFileOutput: true, // Return FileOutput objects (default: true)
      });
      ```
      
      ---
      
      ## Running Predictions with replicate.run()
      
      ### Image Generation
      
      ```typescript
      import Replicate from "replicate";
      
      const replicate = new Replicate();
      
      // Image models return arrays of FileOutput objects
      const [output] = await replicate.run("black-forest-labs/flux-schnell", {
        input: {
          prompt: "a serene mountain landscape at sunset, photorealistic",
          aspect_ratio: "16:9",
          num_outputs: 1,
        },
      });
      
      console.log(output.url()); // Temporary URL to generated image
      ```
      
      ### Text Generation (LLM)
      
      ```typescript
      const output = await replicate.run("meta/meta-llama-3-70b-instruct", {
        input: {
          prompt: "Explain the difference between TCP and UDP in 3 sentences.",
          max_tokens: 256,
          temperature: 0.7,
        },
      });
      
      // LLM output is a string (or array of strings for some models)
      console.log(output);
      ```
      
      ### Audio Transcription
      
      ```typescript
      const output = await replicate.run("openai/whisper", {
        input: {
          audio: "https://example.com/audio-file.mp3",
          model: "large-v3",
          language: "en",
        },
      });
      
      // Output contains transcription text and segments
      console.log(output.text);
      ```
      
      ---
      
      ## Version Pinning
      
      ```typescript
      // Development: latest version (convenient but unpredictable)
      const output = await replicate.run("stability-ai/sdxl", {
        input: { prompt: "a cat" },
      });
      
      // Production: pin to a specific version hash
      const SDXL_VERSION =
        "39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b";
      const output = await replicate.run(`stability-ai/sdxl:${SDXL_VERSION}`, {
        input: { prompt: "a cat" },
      });
      ```
      
      ### Listing Model Versions
      
      ```typescript
      // Get all versions of a model
      const versions = await replicate.models.versions.list("stability-ai", "sdxl");
      for (const version of versions.results) {
        console.log(version.id, version.created_at);
      }
      
      // Get a specific version
      const version = await replicate.models.versions.get(
        "stability-ai",
        "sdxl",
        "39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
      );
      console.log(version.openapi_schema); // Model's input/output schema
      ```
      
      ---
      
      ## Async Predictions with predictions.create()
      
      ```typescript
      // Create a prediction (returns immediately, does not wait for completion)
      // Use `version` for pinned reproducibility, or `model` for latest-version convenience
      const prediction = await replicate.predictions.create({
        model: "owner/model", // OR version: "sha256hash" for pinned version
        input: {
          prompt: "a painting of a cat in the style of Van Gogh",
        },
        webhook: "https://my.app/webhooks/replicate",
        webhook_events_filter: ["completed"],
      });
      
      console.log(prediction.id); // "abc123xyz"
      console.log(prediction.status); // "starting"
      
      // Option 1: Poll until complete
      const result = await replicate.wait(prediction);
      console.log(result.output);
      console.log(result.status); // "succeeded" | "failed" | "canceled"
      
      // Option 2: Check status manually
      const updated = await replicate.predictions.get(prediction.id);
      console.log(updated.status);
      
      // Option 3: Cancel if no longer needed
      await replicate.predictions.cancel(prediction.id);
      ```
      
      ---
      
      ## Progress Callbacks
      
      ```typescript
      import type { Prediction } from "replicate";
      
      function onProgress(prediction: Prediction): void {
        console.log(`Status: ${prediction.status}`);
        if (prediction.logs) {
          console.log(prediction.logs);
        }
      }
      
      const output = await replicate.run(
        "black-forest-labs/flux-schnell",
        { input: { prompt: "a sunset" } },
        onProgress,
      );
      ```
      
      ---
      
      ## File Input Handling
      
      ### URL Input (Recommended for Large Files)
      
      ```typescript
      // Pass a URL -- no upload needed, no size limit
      const output = await replicate.run("owner/image-upscaler", {
        input: {
          image: "https://example.com/photo.jpg",
          scale: 4,
        },
      });
      ```
      
      ### Buffer Input (Auto-Uploaded, Max 100 MiB)
      
      ```typescript
      import { readFile } from "node:fs/promises";
      
      const imageBuffer = await readFile("./input.png");
      
      const output = await replicate.run("owner/image-upscaler", {
        input: {
          image: imageBuffer, // Automatically uploaded to Replicate
          scale: 4,
        },
      });
      ```
      
      ### Explicit File Upload via Files API
      
      ```typescript
      // Upload a file and get a persistent reference
      const file = await replicate.files.create(
        await readFile("./training-data.zip"),
        { filename: "training-data.zip" },
      );
      
      console.log(file.id); // File ID for later reference
      console.log(file.urls); // Temporary download URLs
      ```
      
      ---
      
      ## File Output Handling
      
      ```typescript
      import { writeFile } from "node:fs/promises";
      
      const [output] = await replicate.run("black-forest-labs/flux-schnell", {
        input: { prompt: "a beautiful garden" },
      });
      
      // FileOutput implements ReadableStream
      // Method 1: Get the URL
      const url = output.url();
      console.log(url); // Temporary URL (expires -- download promptly)
      
      // Method 2: Get as Blob, then save
      const blob = await output.blob();
      const buffer = Buffer.from(await blob.arrayBuffer());
      await writeFile("./output.png", buffer);
      
      // Method 3: Write directly using Node.js stream utilities
      await writeFile("./output.png", output);
      ```
      
      ---
      
      ## Error Handling
      
      ```typescript
      import Replicate from "replicate";
      
      const replicate = new Replicate();
      
      async function safePrediction(prompt: string): Promise<string | null> {
        try {
          const output = await replicate.run("meta/meta-llama-3-70b-instruct", {
            input: { prompt, max_tokens: 512 },
          });
      
          return String(output);
        } catch (error) {
          if (!(error instanceof Error)) {
            throw error;
          }
      
          console.error(`Replicate error: ${error.message}`);
      
          // Check for HTTP status codes
          if ("status" in error) {
            const status = (error as { status: number }).status;
      
            switch (status) {
              case 401:
                throw new Error("Invalid API token. Check REPLICATE_API_TOKEN.");
              case 404:
                console.error("Model not found. Check the model identifier.");
                return null;
              case 422:
                console.error("Invalid input parameters for this model.");
                return null;
              case 429:
                console.error("Rate limited. All retries exhausted.");
                return null;
            }
      
            // 5xx errors -- SDK auto-retried, all attempts failed
            if (status >= 500) {
              console.error("Server error after all retries.");
              return null;
            }
          }
      
          throw error; // Re-throw unexpected errors
        }
      }
      
      const result = await safePrediction("Hello!");
      if (result) {
        console.log(result);
      }
      ```
      
      ---
      
      ## Listing Predictions
      
      ```typescript
      // List recent predictions (paginated)
      const page = await replicate.predictions.list();
      
      for (const prediction of page.results) {
        console.log(prediction.id, prediction.status, prediction.model);
      }
      
      // Get next page
      if (page.next) {
        const nextPage = await replicate.predictions.list({ cursor: page.next });
      }
      ```
      
      ---
      
      _For streaming and webhooks, see [streaming-webhooks.md](streaming-webhooks.md). For deployments and training, see [deployments-training.md](deployments-training.md). For API reference tables, see [reference.md](../reference.md)._
      
    • deployments-training.md 6.5 KB
      # Replicate SDK -- Deployments & Training Examples
      
      > Deployment management, custom hardware, scaling configuration, fine-tuning models, and model CRUD operations. See [SKILL.md](../SKILL.md) for core patterns.
      
      **Related examples:**
      
      - [core.md](core.md) -- Client setup, predictions, file handling
      - [streaming-webhooks.md](streaming-webhooks.md) -- Streaming output, webhooks
      
      ---
      
      ## Deployments
      
      ### Why Use Deployments
      
      Deployments give you a **private, fixed API endpoint** with control over:
      
      - **Hardware** -- Choose specific GPU types (e.g., A40, A100)
      - **Scaling** -- Set `min_instances` to keep models warm (eliminates cold starts)
      - **Versioning** -- Pin to a model version independently of the public model page
      
      ### Creating a Deployment
      
      ```typescript
      const deployment = await replicate.deployments.create({
        name: "my-llama-deployment",
        model: "meta/meta-llama-3-70b-instruct",
        version: "abc123...",
        hardware: "gpu-a100-large",
        min_instances: 1, // Always warm -- no cold starts
        max_instances: 5, // Auto-scale up to 5 instances
      });
      
      console.log(deployment.name);
      console.log(deployment.current_release);
      ```
      
      ### Running Predictions on a Deployment
      
      ```typescript
      // Use the deployment name instead of model identifier
      const prediction = await replicate.deployments.predictions.create(
        "my-org/my-llama-deployment",
        {
          input: {
            prompt: "Summarize this article...",
            max_tokens: 1024,
          },
        },
      );
      
      // Wait for result
      const result = await replicate.wait(prediction);
      console.log(result.output);
      ```
      
      ### Streaming from a Deployment
      
      ```typescript
      const stream = replicate.stream("my-org/my-llama-deployment", {
        input: { prompt: "Hello world" },
      });
      
      for await (const event of stream) {
        if (event.event === "output") {
          process.stdout.write(event.data);
        }
      }
      ```
      
      ### Managing Deployments
      
      ```typescript
      // List all deployments
      const deployments = await replicate.deployments.list();
      for (const d of deployments.results) {
        console.log(d.name, d.current_release?.model);
      }
      
      // Get a specific deployment
      const deployment = await replicate.deployments.get(
        "my-org/my-llama-deployment",
      );
      
      // Update deployment (change hardware, version, or scaling)
      await replicate.deployments.update("my-org/my-llama-deployment", {
        version: "new-version-hash",
        hardware: "gpu-a100-large",
        min_instances: 2,
        max_instances: 10,
      });
      
      // Delete a deployment
      await replicate.deployments.delete("my-org/my-llama-deployment");
      ```
      
      ### Available Hardware
      
      ```typescript
      const hardware = await replicate.hardware.list();
      for (const hw of hardware) {
        console.log(hw.name, hw.sku);
      }
      ```
      
      See [reference.md](../reference.md) for the full hardware SKU table.
      
      ---
      
      ## Training (Fine-Tuning)
      
      ### Starting a Training Job
      
      ```typescript
      const training = await replicate.trainings.create(
        "owner",
        "model-name",
        "version-hash",
        {
          input: {
            train_data: "https://example.com/my-training-data.zip",
            num_train_epochs: 4,
            learning_rate: 0.0001,
          },
          destination: "my-org/my-fine-tuned-model",
          webhook: "https://my.app/webhooks/training",
          webhook_events_filter: ["completed"],
        },
      );
      
      console.log(training.id);
      console.log(training.status); // "starting"
      ```
      
      ### Monitoring Training Progress
      
      ```typescript
      // Poll for training status
      const training = await replicate.trainings.get(training.id);
      console.log(training.status); // "starting" | "processing" | "succeeded" | "failed"
      console.log(training.logs); // Training logs
      
      // Wait for completion
      const result = await replicate.wait(training);
      
      if (result.status === "succeeded") {
        console.log("Training complete!");
        console.log("New model version:", result.output?.version);
      } else {
        console.error("Training failed:", result.error);
      }
      ```
      
      ### Listing and Canceling Trainings
      
      ```typescript
      // List all trainings
      const trainings = await replicate.trainings.list();
      for (const t of trainings.results) {
        console.log(t.id, t.status, t.model);
      }
      
      // Cancel a running training
      await replicate.trainings.cancel(training.id);
      ```
      
      ---
      
      ## Model Management
      
      ### Getting Model Info
      
      ```typescript
      const model = await replicate.models.get("stability-ai", "sdxl");
      
      console.log(model.owner);
      console.log(model.name);
      console.log(model.description);
      console.log(model.visibility); // "public" | "private"
      console.log(model.latest_version?.id);
      ```
      
      ### Creating a Model
      
      ```typescript
      // Create a new model (for training destinations or custom models)
      const model = await replicate.models.create("my-org", "my-custom-model", {
        description: "A fine-tuned image generation model",
        visibility: "private",
        hardware: "gpu-a40-large",
      });
      ```
      
      ### Listing Models
      
      ```typescript
      // List your models
      const models = await replicate.models.list();
      for (const m of models.results) {
        console.log(m.owner, m.name, m.run_count);
      }
      
      // Search public models
      const results = await replicate.models.search("text to image");
      for (const m of results.results) {
        console.log(m.owner, m.name, m.description);
      }
      ```
      
      ### Model Versions
      
      ```typescript
      // List all versions
      const versions = await replicate.models.versions.list("stability-ai", "sdxl");
      for (const v of versions.results) {
        console.log(v.id, v.created_at);
      }
      
      // Get a specific version (includes input/output schema)
      const version = await replicate.models.versions.get(
        "stability-ai",
        "sdxl",
        "39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
      );
      
      // Access input/output schema from OpenAPI spec
      console.log(version.openapi_schema);
      ```
      
      ---
      
      ## Collections
      
      ```typescript
      // List model collections
      const collections = await replicate.collections.list();
      for (const c of collections.results) {
        console.log(c.slug, c.name, c.description);
      }
      
      // Get a specific collection
      const collection = await replicate.collections.get("text-to-image");
      for (const model of collection.models) {
        console.log(model.owner, model.name);
      }
      ```
      
      ---
      
      ## Files API
      
      ```typescript
      import { readFile } from "node:fs/promises";
      
      // Upload a file
      const file = await replicate.files.create(await readFile("./data.zip"), {
        filename: "data.zip",
      });
      console.log(file.id);
      
      // List files
      const files = await replicate.files.list();
      for (const f of files.results) {
        console.log(f.id, f.name, f.size);
      }
      
      // Get file info
      const fileInfo = await replicate.files.get(file.id);
      
      // Delete a file
      await replicate.files.delete(file.id);
      ```
      
      ---
      
      _For client setup and predictions, see [core.md](core.md). For streaming and webhooks, see [streaming-webhooks.md](streaming-webhooks.md). For API reference tables, see [reference.md](../reference.md)._
      
    • streaming-webhooks.md 7.2 KB
      # Replicate SDK -- Streaming & Webhooks Examples
      
      > Streaming output with SSE events, webhook setup, event filtering, and signature validation. See [SKILL.md](../SKILL.md) for core patterns.
      
      **Related examples:**
      
      - [core.md](core.md) -- Client setup, predictions, file handling
      - [deployments-training.md](deployments-training.md) -- Deployments, training, model management
      
      ---
      
      ## Basic Streaming with replicate.stream()
      
      ```typescript
      import Replicate from "replicate";
      
      const replicate = new Replicate();
      
      const stream = replicate.stream("meta/meta-llama-3-70b-instruct", {
        input: {
          prompt: "Explain async/await in TypeScript.",
          max_tokens: 512,
        },
      });
      
      for await (const event of stream) {
        if (event.event === "output") {
          process.stdout.write(event.data);
        }
      }
      console.log(); // newline
      ```
      
      ---
      
      ## Handling All SSE Event Types
      
      ```typescript
      const stream = replicate.stream("meta/meta-llama-3-70b-instruct", {
        input: { prompt: "Tell me a story." },
      });
      
      const chunks: string[] = [];
      
      for await (const event of stream) {
        switch (event.event) {
          case "output":
            // Progressive text output
            chunks.push(event.data);
            process.stdout.write(event.data);
            break;
      
          case "error":
            // Prediction error (event.data is JSON with details)
            console.error("Stream error:", event.data);
            break;
      
          case "done":
            // Prediction complete (event.data may contain reason)
            console.log("\nStream complete");
            break;
        }
      }
      
      const fullOutput = chunks.join("");
      console.log("Total length:", fullOutput.length);
      ```
      
      ### SSE Event Types Reference
      
      | Event    | Data Format | Description                                                |
      | -------- | ----------- | ---------------------------------------------------------- |
      | `output` | Plain text  | New model output chunk                                     |
      | `error`  | JSON string | Error details (e.g., `{"detail": "..."}`)                  |
      | `done`   | JSON string | Completion signal (e.g., `{}` or `{"reason": "canceled"}`) |
      
      ---
      
      ## Streaming with Error Handling
      
      ```typescript
      try {
        const stream = replicate.stream("meta/meta-llama-3-70b-instruct", {
          input: { prompt: "Hello" },
        });
      
        for await (const event of stream) {
          if (event.event === "output") {
            process.stdout.write(event.data);
          }
          if (event.event === "error") {
            throw new Error(`Prediction error: ${event.data}`);
          }
        }
      } catch (error) {
        if (error instanceof Error) {
          console.error("Stream failed:", error.message);
        }
        throw error;
      }
      ```
      
      ---
      
      ## Streaming on Deployments
      
      ```typescript
      // Stream from a deployment (consistent latency if min_instances > 0)
      const stream = replicate.stream("my-org/my-deployment", {
        input: { prompt: "Summarize this document..." },
      });
      
      for await (const event of stream) {
        if (event.event === "output") {
          process.stdout.write(event.data);
        }
      }
      ```
      
      ---
      
      ## Webhook Setup
      
      ### Creating a Prediction with Webhooks
      
      ```typescript
      const prediction = await replicate.predictions.create({
        version: "27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
        input: { prompt: "a painting of a sunset" },
        webhook: "https://my.app/webhooks/replicate",
        webhook_events_filter: ["completed"],
      });
      
      console.log(prediction.id); // Track via ID
      ```
      
      ### Webhook Events Filter Options
      
      | Event       | Description                               | When to Use                              |
      | ----------- | ----------------------------------------- | ---------------------------------------- |
      | `start`     | Prediction has started processing         | Track cold start / queue time            |
      | `output`    | New output is available (intermediate)    | Progressive updates for long predictions |
      | `logs`      | New log output from the model             | Debugging, monitoring                    |
      | `completed` | Prediction finished (succeeded or failed) | Most common -- final result notification |
      
      ```typescript
      // Receive all events (verbose, useful for debugging)
      const prediction = await replicate.predictions.create({
        version: "abc123...",
        input: { prompt: "hello" },
        webhook: "https://my.app/webhooks/replicate",
        webhook_events_filter: ["start", "output", "logs", "completed"],
      });
      
      // Receive only the final result (recommended for production)
      const prediction = await replicate.predictions.create({
        version: "abc123...",
        input: { prompt: "hello" },
        webhook: "https://my.app/webhooks/replicate",
        webhook_events_filter: ["completed"],
      });
      ```
      
      ---
      
      ## Webhook Signature Validation
      
      ### Using the Built-in Validator
      
      ```typescript
      import { validateWebhook } from "replicate";
      
      async function handleWebhook(request: Request): Promise<Response> {
        const secret = process.env.REPLICATE_WEBHOOK_SIGNING_SECRET;
      
        if (!secret) {
          throw new Error("REPLICATE_WEBHOOK_SIGNING_SECRET not set");
        }
      
        const isValid = await validateWebhook(request, secret);
      
        if (!isValid) {
          return new Response("Invalid webhook signature", { status: 401 });
        }
      
        const prediction = await request.json();
      
        // Safe to process -- signature verified
        console.log("Prediction status:", prediction.status);
        console.log("Prediction output:", prediction.output);
      
        if (prediction.status === "succeeded") {
          // Handle successful prediction
          await processOutput(prediction.output);
        } else if (prediction.status === "failed") {
          console.error("Prediction failed:", prediction.error);
        }
      
        return new Response("OK", { status: 200 });
      }
      ```
      
      ### Manual Signature Validation
      
      ```typescript
      import { validateWebhook } from "replicate";
      
      // When you have raw request data instead of a Request object
      const isValid = await validateWebhook({
        id: request.headers["webhook-id"],
        timestamp: request.headers["webhook-timestamp"],
        signature: request.headers["webhook-signature"],
        body: rawBody, // string | ArrayBuffer | ReadableStream
        secret: process.env.REPLICATE_WEBHOOK_SIGNING_SECRET,
      });
      ```
      
      ---
      
      ## Webhook Payload Structure
      
      ```typescript
      // The webhook POST body is the same as the prediction object
      interface WebhookPayload {
        id: string;
        model: string;
        version: string;
        status: "starting" | "processing" | "succeeded" | "failed" | "canceled";
        input: Record<string, unknown>;
        output: unknown; // Model-specific output
        error: string | null;
        logs: string;
        created_at: string;
        started_at: string | null;
        completed_at: string | null;
        urls: {
          get: string;
          cancel: string;
          stream?: string;
        };
      }
      ```
      
      ---
      
      ## Polling as Alternative to Webhooks
      
      ```typescript
      // For environments where webhooks are not feasible
      const prediction = await replicate.predictions.create({
        version: "abc123...",
        input: { prompt: "hello" },
      });
      
      // replicate.wait() polls until the prediction completes
      const result = await replicate.wait(prediction);
      
      if (result.status === "succeeded") {
        console.log(result.output);
      } else if (result.status === "failed") {
        console.error("Failed:", result.error);
      } else if (result.status === "canceled") {
        console.log("Prediction was canceled");
      }
      ```
      
      ---
      
      _For client setup and file handling, see [core.md](core.md). For deployments and training, see [deployments-training.md](deployments-training.md). For API reference tables, see [reference.md](../reference.md)._
      
  • reference.md 9.1 KB
    # Replicate SDK Quick Reference
    
    > Constructor options, API methods, error types, model reference format, and webhook events. See [SKILL.md](SKILL.md) for core concepts and [examples/](examples/) for code examples.
    
    ---
    
    ## Package Installation
    
    ```bash
    npm install replicate
    ```
    
    ---
    
    ## Client Configuration
    
    ```typescript
    import Replicate from "replicate";
    
    const replicate = new Replicate({
      auth: process.env.REPLICATE_API_TOKEN, // Auto-reads from env if omitted
      baseUrl: "https://api.replicate.com/v1", // Override for proxies
      userAgent: "my-app/1.0.0", // Custom user agent string
      fetch: globalThis.fetch, // Custom fetch implementation (Node.js 18+)
      fileEncodingStrategy: "default", // "default" | "upload" | "data-uri"
      useFileOutput: true, // Return FileOutput objects for file outputs (default: true)
    });
    ```
    
    ### Environment Variables
    
    | Variable                           | Purpose                             |
    | ---------------------------------- | ----------------------------------- |
    | `REPLICATE_API_TOKEN`              | API token (auto-detected)           |
    | `REPLICATE_WEBHOOK_SIGNING_SECRET` | Webhook signature validation secret |
    
    ---
    
    ## Model Reference Format
    
    ```
    owner/model                          -- Uses latest version (convenient, not reproducible)
    owner/model:sha256hash               -- Pinned to a specific version (production use)
    ```
    
    ### Examples
    
    | Reference                        | Type                     |
    | -------------------------------- | ------------------------ |
    | `meta/meta-llama-3-70b-instruct` | Latest version           |
    | `black-forest-labs/flux-schnell` | Latest version           |
    | `stability-ai/sdxl:39ed52f2...`  | Pinned version (SHA-256) |
    
    ---
    
    ## API Methods Reference
    
    ### Core Methods
    
    ```typescript
    // Run a model synchronously (waits for completion)
    const output = await replicate.run(
      "owner/model",               // or "owner/model:version"
      {
        input: { prompt: "..." },  // Model-specific input parameters
      },
      onProgress?,                 // Optional: (prediction: Prediction) => void
    );
    
    // Stream model output as SSE events
    const stream = replicate.stream(
      "owner/model",               // or deployment name
      {
        input: { prompt: "..." },  // Model-specific input parameters
      },
    );
    // Returns AsyncGenerator<ServerSentEvent>
    
    // Wait for a prediction to complete (polling)
    const result = await replicate.wait(prediction);
    ```
    
    ### Predictions
    
    ```typescript
    // Create (async, returns immediately)
    // Specify EITHER model OR version (not both)
    const prediction = await replicate.predictions.create({
      model: "owner/name",                      // Use for latest version
      // OR: version: "sha256hash",             // Use for pinned reproducibility
      input: { prompt: "..." },                 // Required: model inputs
      webhook?: "https://...",                   // Optional: webhook URL
      webhook_events_filter?: ["completed"],     // Optional: event filter
    });
    
    // Get prediction status
    const prediction = await replicate.predictions.get(prediction.id);
    
    // List predictions (paginated)
    const page = await replicate.predictions.list();
    
    // Cancel a running prediction
    await replicate.predictions.cancel(prediction.id);
    ```
    
    ### Models
    
    ```typescript
    // Get model info
    const model = await replicate.models.get("owner", "name");
    
    // List models (paginated)
    const models = await replicate.models.list();
    
    // Search models
    const results = await replicate.models.search("query");
    
    // Create a model
    await replicate.models.create("owner", "name", {
      description?: string,
      visibility: "public" | "private",
      hardware?: string,
    });
    
    // List model versions
    const versions = await replicate.models.versions.list("owner", "name");
    
    // Get a specific version
    const version = await replicate.models.versions.get("owner", "name", "version_id");
    ```
    
    ### Deployments
    
    ```typescript
    // Create a deployment
    await replicate.deployments.create({
      name: string,
      model: string,
      version: string,
      hardware: string,
      min_instances?: number,
      max_instances?: number,
    });
    
    // Run prediction on deployment
    await replicate.deployments.predictions.create("owner/deployment", {
      input: { ... },
      webhook?: string,
    });
    
    // Get deployment info
    await replicate.deployments.get("owner/deployment");
    
    // List deployments
    await replicate.deployments.list();
    
    // Update deployment
    await replicate.deployments.update("owner/deployment", { ... });
    
    // Delete deployment
    await replicate.deployments.delete("owner/deployment");
    ```
    
    ### Trainings
    
    ```typescript
    // Start training
    await replicate.trainings.create("owner", "model", "version", {
      input: { train_data: "https://...", ... },
      destination: "owner/new-model",
      webhook?: string,
      webhook_events_filter?: string[],
    });
    
    // Get training status
    await replicate.trainings.get(training.id);
    
    // List trainings
    await replicate.trainings.list();
    
    // Cancel training
    await replicate.trainings.cancel(training.id);
    ```
    
    ### Files
    
    ```typescript
    // Upload file
    await replicate.files.create(buffer, { filename: "data.zip" });
    
    // Get file info
    await replicate.files.get(file.id);
    
    // List files
    await replicate.files.list();
    
    // Delete file
    await replicate.files.delete(file.id);
    ```
    
    ### Hardware
    
    ```typescript
    const hardware = await replicate.hardware.list();
    // Returns: Array<{ name: string, sku: string }>
    ```
    
    ### Collections
    
    ```typescript
    await replicate.collections.get("collection-slug");
    await replicate.collections.list();
    ```
    
    ---
    
    ## Prediction Status Values
    
    | Status       | Description                          |
    | ------------ | ------------------------------------ |
    | `starting`   | Prediction is queued / model booting |
    | `processing` | Model is running                     |
    | `succeeded`  | Prediction completed successfully    |
    | `failed`     | Prediction encountered an error      |
    | `canceled`   | Prediction was canceled by the user  |
    
    ---
    
    ## SSE Event Types (Streaming)
    
    | Event    | Data Format | Description                                                |
    | -------- | ----------- | ---------------------------------------------------------- |
    | `output` | Plain text  | New model output chunk                                     |
    | `error`  | JSON string | Error details (e.g., `{"detail": "Something went wrong"}`) |
    | `done`   | JSON string | Completion signal (e.g., `{}` or `{"reason": "canceled"}`) |
    
    ---
    
    ## Webhook Events
    
    | Event       | Description                               |
    | ----------- | ----------------------------------------- |
    | `start`     | Prediction has started processing         |
    | `output`    | New output is available (intermediate)    |
    | `logs`      | New log output from the model             |
    | `completed` | Prediction finished (succeeded or failed) |
    
    Webhook payload is the same as the prediction object (JSON POST to your URL).
    
    ---
    
    ## Webhook Signature Validation
    
    ```typescript
    import { validateWebhook } from "replicate";
    
    // With a Request object
    const isValid = await validateWebhook(request, secret);
    
    // With raw data
    const isValid = await validateWebhook({
      id: headers["webhook-id"],
      timestamp: headers["webhook-timestamp"],
      signature: headers["webhook-signature"],
      body: rawBody,
      secret: process.env.REPLICATE_WEBHOOK_SIGNING_SECRET,
    });
    ```
    
    ---
    
    ## FileOutput Object
    
    ```typescript
    // Returned by replicate.run() for models that output files
    interface FileOutput extends ReadableStream {
      url(): URL; // Get the underlying URL object (temporary -- download promptly)
      blob(): Promise<Blob>; // Get as Blob
      toString(): string; // String representation
    }
    ```
    
    ---
    
    ## Error Types
    
    The SDK throws errors with HTTP status codes for API failures:
    
    | Status | Description          | Auto-Retried? |
    | ------ | -------------------- | ------------- |
    | 400    | Bad Request          | No            |
    | 401    | Authentication Error | No            |
    | 403    | Permission Denied    | No            |
    | 404    | Not Found            | No            |
    | 422    | Unprocessable Entity | No            |
    | 429    | Rate Limit Exceeded  | Yes           |
    | >= 500 | Server Error         | Yes           |
    
    The SDK automatically retries on 429 and 5xx errors (5 retries by default with exponential backoff). GET requests retry on 429 and 5xx; non-GET requests retry only on 429.
    
    ---
    
    ## Hardware SKUs
    
    | SKU              | Description        |
    | ---------------- | ------------------ |
    | `cpu`            | CPU only           |
    | `gpu-t4-nano`    | Nvidia T4 (small)  |
    | `gpu-t4-small`   | Nvidia T4          |
    | `gpu-a40-small`  | Nvidia A40 (small) |
    | `gpu-a40-large`  | Nvidia A40 (large) |
    | `gpu-a100-large` | Nvidia A100 (80GB) |
    
    ---
    
    ## Platform Support
    
    | Platform            | Minimum Version                          |
    | ------------------- | ---------------------------------------- |
    | Node.js             | 18+                                      |
    | Bun                 | 1.0+                                     |
    | Deno                | 1.28+                                    |
    | Cloudflare Workers  | Supported                                |
    | Vercel Edge Runtime | Supported                                |
    | AWS Lambda          | Supported                                |
    | **Browsers**        | **Not supported** -- use a backend proxy |
    
  • SKILL.md 19.6 KB
    ---
    name: ai-infrastructure-replicate
    description: Replicate SDK patterns for TypeScript/Node.js -- client setup, predictions, streaming, webhooks, file handling, model versioning, deployments, and training
    ---
    
    # Replicate SDK Patterns
    
    > **Quick Guide:** Use the `replicate` npm package to run open-source ML models on serverless GPUs. Use `replicate.run()` for synchronous execution that returns output directly, `replicate.stream()` for SSE-based streaming, or `replicate.predictions.create()` for async background jobs with webhook notifications. Models are referenced as `owner/model` (uses latest version) or `owner/model:version` (pinned). File outputs are `FileOutput` objects implementing `ReadableStream`. Cold starts are expected for infrequently-used models -- use deployments with `min_instances` to keep models warm.
    
    ---
    
    <critical_requirements>
    
    ## CRITICAL: Before Using This Skill
    
    > **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants)
    
    **(You MUST never hardcode API tokens -- always use environment variables via `process.env.REPLICATE_API_TOKEN`)**
    
    **(You MUST handle `FileOutput` objects for models that return files -- do not assume outputs are plain strings or URLs)**
    
    **(You MUST validate webhooks using `validateWebhook()` from the `replicate` package -- never trust unverified webhook payloads)**
    
    **(You MUST account for cold starts when running infrequently-used models -- use deployments for latency-sensitive applications)**
    
    **(You MUST specify model versions (`owner/model:version`) in production to ensure reproducible results -- unversioned references use the latest, which can change)**
    
    </critical_requirements>
    
    ---
    
    **Auto-detection:** Replicate, replicate, replicate.run, replicate.stream, replicate.predictions, replicate.deployments, replicate.trainings, replicate.models, FileOutput, validateWebhook, REPLICATE_API_TOKEN, serverless GPU, cold start, webhook_events_filter
    
    **When to use:**
    
    - Running open-source ML models (Llama, Stable Diffusion, Whisper, etc.) without managing GPU infrastructure
    - Generating images, transcribing audio, running LLMs, or any ML inference via API
    - Streaming LLM output in real-time with server-sent events
    - Processing predictions asynchronously with webhook notifications
    - Fine-tuning models with custom training data
    - Running models on dedicated hardware with custom scaling via deployments
    
    **Key patterns covered:**
    
    - Client initialization and configuration (auth, user agent, file encoding)
    - Running predictions (`replicate.run()`, `replicate.predictions.create()`, `replicate.wait()`)
    - Streaming output (`replicate.stream()` with SSE events)
    - Model versioning (`owner/model` vs `owner/model:version`)
    - File input/output handling (`FileOutput`, file uploads, `Buffer` inputs)
    - Webhooks (setup, event filtering, signature validation)
    - Deployments (custom hardware, scaling, keeping models warm)
    - Training / fine-tuning
    
    **When NOT to use:**
    
    - You need a unified multi-provider LLM SDK (OpenAI, Anthropic, Google) -- use a provider-agnostic SDK
    - You want to run models locally -- Replicate is a cloud-only serverless platform
    - You need sub-second latency guarantees without deployments -- cold starts can take minutes
    
    ---
    
    ## Examples Index
    
    - [Core: Setup, Predictions & Files](examples/core.md) -- Client init, run(), predictions.create(), wait(), file I/O, error handling
    - [Streaming & Webhooks](examples/streaming-webhooks.md) -- stream(), SSE events, webhook setup, signature validation
    - [Deployments & Training](examples/deployments-training.md) -- Custom hardware, scaling, fine-tuning, model management
    - [Quick API Reference](reference.md) -- Method signatures, constructor options, error types, model reference format
    
    ---
    
    <philosophy>
    
    ## Philosophy
    
    Replicate provides **serverless GPU infrastructure** for running open-source ML models. You send inputs, Replicate allocates GPU hardware, runs the model, and returns outputs. No Docker, no CUDA drivers, no GPU provisioning.
    
    **Core principles:**
    
    1. **Serverless execution** -- Models run on-demand on Replicate's infrastructure. You pay only for compute time. Cold starts are a trade-off for not maintaining always-on GPUs.
    2. **Model marketplace** -- Thousands of community and official models available at `replicate.com/explore`. Run any public model with just its identifier.
    3. **Version pinning for reproducibility** -- Models are versioned with SHA-256 hashes. Pin to a version in production (`owner/model:abc123...`) to guarantee identical behavior across deploys.
    4. **Three execution modes** -- `replicate.run()` for synchronous wait, `replicate.stream()` for real-time SSE output, `replicate.predictions.create()` for fire-and-forget with webhooks.
    5. **File-first I/O** -- Many models accept and produce files (images, audio, video). The SDK handles file uploads automatically and returns `FileOutput` objects for file outputs.
    
    </philosophy>
    
    ---
    
    <patterns>
    
    ## Core Patterns
    
    ### Pattern 1: Client Setup
    
    Initialize the Replicate client. It auto-reads `REPLICATE_API_TOKEN` from the environment.
    
    ```typescript
    // lib/replicate.ts -- basic setup
    import Replicate from "replicate";
    
    const replicate = new Replicate();
    
    export { replicate };
    ```
    
    ```typescript
    // lib/replicate.ts -- explicit auth + custom user agent
    import Replicate from "replicate";
    
    const replicate = new Replicate({
      auth: process.env.REPLICATE_API_TOKEN, // Auto-reads from env if omitted
      userAgent: "my-app/1.0.0",
    });
    
    export { replicate };
    ```
    
    **Why good:** Minimal setup, env var auto-detected, explicit auth optional but useful for clarity
    
    ```typescript
    // BAD: Hardcoded token
    const replicate = new Replicate({
      auth: "r8_abc123...",
    });
    ```
    
    **Why bad:** Hardcoded API token is a security risk, will leak in version control
    
    **See:** [examples/core.md](examples/core.md) for full constructor options, error handling patterns
    
    ---
    
    ### Pattern 2: Running Predictions
    
    Use `replicate.run()` for synchronous execution. Returns the model output directly.
    
    ```typescript
    // Run an image generation model
    const [output] = await replicate.run("black-forest-labs/flux-schnell", {
      input: {
        prompt: "a serene mountain landscape at sunset",
      },
    });
    
    // output is a FileOutput object for image models
    console.log(output.url()); // URL of generated image
    ```
    
    ```typescript
    // Run an LLM -- output is a string for text models
    const output = await replicate.run("meta/meta-llama-3-70b-instruct", {
      input: {
        prompt: "Explain TypeScript generics in 3 sentences.",
        max_tokens: 512,
      },
    });
    
    console.log(output); // Text response
    ```
    
    **Why good:** Simple API, returns output directly, destructuring works for array outputs (images)
    
    ```typescript
    // BAD: Not pinning version in production
    const output = await replicate.run("community-user/experimental-model", {
      input: { prompt: "hello" },
    });
    ```
    
    **Why bad:** Community models without version pinning can change behavior unexpectedly when authors push updates
    
    **See:** [examples/core.md](examples/core.md) for version pinning, `predictions.create()` + `wait()`, and progress callbacks
    
    ---
    
    ### Pattern 3: Streaming
    
    Use `replicate.stream()` for real-time SSE output from language models.
    
    ```typescript
    const stream = replicate.stream("meta/meta-llama-3-70b-instruct", {
      input: {
        prompt: "Write a short poem about TypeScript.",
        max_tokens: 512,
      },
    });
    
    for await (const event of stream) {
      if (event.event === "output") {
        process.stdout.write(event.data);
      }
    }
    ```
    
    **Why good:** Progressive output for better UX, event-based with typed `event` and `data` fields
    
    ```typescript
    // BAD: Using replicate.run() for user-facing LLM output
    const output = await replicate.run("meta/meta-llama-3-70b-instruct", {
      input: { prompt: "Write a long essay..." },
    });
    // User waits for entire generation to complete before seeing anything
    ```
    
    **Why bad:** No progressive feedback, user sees a blank screen for seconds
    
    **See:** [examples/streaming-webhooks.md](examples/streaming-webhooks.md) for event types, error handling, cancellation
    
    ---
    
    ### Pattern 4: Model Versioning
    
    Models are referenced as `owner/model` (latest version) or `owner/model:sha256hash` (pinned version).
    
    ```typescript
    // Development: use latest version for convenience
    const output = await replicate.run("stability-ai/sdxl", {
      input: { prompt: "a cat" },
    });
    
    // Production: pin to a specific version for reproducibility
    const VERSION_HASH =
      "39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b";
    const output = await replicate.run(`stability-ai/sdxl:${VERSION_HASH}`, {
      input: { prompt: "a cat" },
    });
    ```
    
    **Why good:** Pinned version guarantees identical behavior, hash is immutable
    
    **See:** [examples/core.md](examples/core.md) for listing model versions, getting version details
    
    ---
    
    ### Pattern 5: File Handling
    
    Models that output files return `FileOutput` objects implementing `ReadableStream`.
    
    ```typescript
    import { writeFile } from "node:fs/promises";
    
    const [output] = await replicate.run("black-forest-labs/flux-schnell", {
      input: { prompt: "a sunset over mountains" },
    });
    
    // FileOutput has .url() and .blob() methods
    console.log(output.url()); // Underlying URL
    
    // Save to disk
    const blob = await output.blob();
    const buffer = Buffer.from(await blob.arrayBuffer());
    await writeFile("./output.png", buffer);
    ```
    
    ```typescript
    // File inputs: pass URLs, Buffers, or ReadStreams
    import { readFile } from "node:fs/promises";
    
    const imageBuffer = await readFile("./input.png");
    
    const output = await replicate.run("some-user/image-model", {
      input: {
        image: imageBuffer, // Auto-uploaded (max 100 MiB)
      },
    });
    ```
    
    **Why good:** `FileOutput` is a `ReadableStream`, works with Node.js stream APIs, `.url()` for the underlying URL
    
    ```typescript
    // BAD: Treating file output as a plain URL string
    const [output] = await replicate.run("black-forest-labs/flux-schnell", {
      input: { prompt: "hello" },
    });
    const url = output; // WRONG: output is a FileOutput object, not a string
    ```
    
    **Why bad:** `FileOutput` is an object, not a string -- use `.url()` to get the URL
    
    **See:** [examples/core.md](examples/core.md) for file uploads, large file handling, encoding strategies
    
    ---
    
    ### Pattern 6: Async Predictions with Webhooks
    
    Use `replicate.predictions.create()` for background jobs with webhook notifications.
    
    ```typescript
    const prediction = await replicate.predictions.create({
      model: "owner/model", // OR version: "sha256hash" for pinned version
      input: { prompt: "a painting of a cat" },
      webhook: "https://my.app/webhooks/replicate",
      webhook_events_filter: ["completed"],
    });
    
    console.log(prediction.id); // Use to track status
    console.log(prediction.status); // "starting"
    ```
    
    ```typescript
    // Webhook signature validation (CRITICAL for security)
    import { validateWebhook } from "replicate";
    
    async function handleWebhook(request: Request): Promise<Response> {
      const secret = process.env.REPLICATE_WEBHOOK_SIGNING_SECRET;
      const isValid = await validateWebhook(request, secret);
    
      if (!isValid) {
        return new Response("Invalid signature", { status: 401 });
      }
    
      const prediction = await request.json();
      // Process prediction.output safely
      return new Response("OK", { status: 200 });
    }
    ```
    
    **Why good:** Decoupled processing, secure signature validation, filtered events reduce noise
    
    **See:** [examples/streaming-webhooks.md](examples/streaming-webhooks.md) for webhook event types, polling alternative
    
    ---
    
    ### Pattern 7: Deployments
    
    Deployments give you a private, fixed endpoint with custom hardware and scaling.
    
    ```typescript
    // Create a prediction on a deployment (no cold start if min_instances > 0)
    const prediction = await replicate.deployments.predictions.create(
      "my-org/my-deployment",
      {
        input: { prompt: "hello world" },
      },
    );
    
    const result = await replicate.wait(prediction);
    console.log(result.output);
    ```
    
    **Why good:** Predictable latency with `min_instances`, private endpoint, custom hardware selection
    
    **See:** [examples/deployments-training.md](examples/deployments-training.md) for creating/managing deployments, training API
    
    ---
    
    ### Pattern 8: Error Handling
    
    Catch API errors with status codes. The SDK auto-retries on 429 and 5xx errors (5 retries by default with exponential backoff).
    
    ```typescript
    try {
      const output = await replicate.run("owner/model", {
        input: { prompt: "hello" },
      });
    } catch (error) {
      if (error instanceof Error) {
        console.error(`Replicate error: ${error.message}`);
    
        // Check for specific HTTP status codes in the error
        if ("status" in error) {
          const status = (error as { status: number }).status;
          if (status === 401) {
            throw new Error("Invalid API token. Check REPLICATE_API_TOKEN.");
          }
          if (status === 422) {
            console.error("Invalid input parameters");
          }
          if (status === 429) {
            console.error(
              "Rate limited -- SDK auto-retries (5 attempts) exhausted",
            );
          }
        }
      }
      throw error;
    }
    ```
    
    **Why good:** Checks error type, handles specific status codes, re-throws unexpected errors
    
    **See:** [examples/core.md](examples/core.md) for full error handling example with status code handling
    
    </patterns>
    
    ---
    
    <performance>
    
    ## Performance Optimization
    
    ### Cold Start Mitigation
    
    ```
    Frequent model with varying load   -> Use deployments with min_instances >= 1
    One-off batch jobs                  -> Use predictions.create() with webhooks (no waiting)
    Popular public models               -> Usually warm, replicate.run() is fine
    Custom/niche models                 -> Expect 30s-5min cold start on first run
    ```
    
    ### Key Optimization Patterns
    
    - **Use deployments** for latency-sensitive applications -- set `min_instances: 1` to eliminate cold starts
    - **Use webhooks** instead of polling for async jobs -- reduces API calls and latency
    - **Batch file inputs as URLs** instead of uploading buffers -- avoids 100 MiB upload limit and is faster
    - **Pin model versions** in production -- avoids unexpected behavior changes and enables caching
    - **Use `replicate.stream()`** for LLMs -- progressive output feels faster than waiting for full completion
    - **Cancel unneeded predictions** with `replicate.predictions.cancel()` -- stops billing immediately
    
    </performance>
    
    ---
    
    <decision_framework>
    
    ## Decision Framework
    
    ### Which Execution Method to Use
    
    ```
    Is this a user-facing LLM response?
    +-- YES -> Use replicate.stream() for real-time SSE output
    +-- NO -> Do you need the result immediately?
        +-- YES -> Use replicate.run() (blocks until complete)
        +-- NO -> Use replicate.predictions.create() + webhook
            +-- Need to poll instead? -> Use replicate.wait(prediction)
    ```
    
    ### Model Reference Format
    
    ```
    Are you in development/prototyping?
    +-- YES -> Use owner/model (latest version, convenient)
    +-- NO -> Are you in production?
        +-- YES -> Use owner/model:version_hash (pinned, reproducible)
        +-- Does the model change frequently?
            +-- YES -> Pin version, test updates explicitly
            +-- NO -> Either format works, prefer pinned
    ```
    
    ### Deployments vs Direct API
    
    ```
    Do you need consistent low latency?
    +-- YES -> Create a deployment with min_instances >= 1
    +-- NO -> Do you need custom hardware (A100, H100)?
        +-- YES -> Create a deployment with specific hardware
        +-- NO -> Use replicate.run() / replicate.stream() directly
            (Replicate auto-allocates hardware)
    ```
    
    ### When to Use This SDK vs Other AI SDKs
    
    ```
    Are you running open-source models on serverless GPUs?
    +-- YES -> Use Replicate SDK
    +-- NO -> Are you calling proprietary APIs (OpenAI, Anthropic)?
        +-- YES -> Not this skill's scope -- use provider-specific SDKs
        +-- NO -> Do you need to switch between multiple providers?
            +-- YES -> Not this skill's scope -- use a unified provider SDK
            +-- NO -> Do you want to self-host models?
                +-- YES -> Not this skill's scope -- consider Cog or vLLM
                +-- NO -> Replicate SDK is appropriate
    ```
    
    </decision_framework>
    
    ---
    
    <red_flags>
    
    ## RED FLAGS
    
    **High Priority Issues:**
    
    - Hardcoding `REPLICATE_API_TOKEN` in source code (security breach risk)
    - Treating `FileOutput` as a string (it is a `ReadableStream` object -- use `.url()` or `.blob()`)
    - Not validating webhook signatures with `validateWebhook()` (allows forged webhook payloads)
    - Using `replicate.run()` for long-running models in request handlers (blocks the response, can timeout)
    
    **Medium Priority Issues:**
    
    - Not pinning model versions in production (`owner/model` uses latest, which can change without notice)
    - Relying solely on default retry behavior for production (5 retries with exponential backoff may be too aggressive for some use cases)
    - Uploading large files as `Buffer` instead of hosting them at a URL (100 MiB limit on uploads)
    - Ignoring cold start latency for infrequently-used models (first request can take minutes)
    
    **Common Mistakes:**
    
    - Confusing `replicate.run()` (returns output directly) with `replicate.predictions.create()` (returns a prediction object with status/id)
    - Destructuring image output incorrectly: `const output = await replicate.run(...)` instead of `const [output] = await replicate.run(...)` (image models return arrays)
    - Using `replicate.stream()` with models that do not support streaming (only language models with SSE support)
    - Forgetting that `replicate.predictions.create()` accepts either a `version` hash or a `model` string (`owner/model`) -- use `version` for pinned reproducibility, `model` for latest-version convenience
    - Not consuming the async iterator from `replicate.stream()` (events are lost)
    
    **Gotchas & Edge Cases:**
    
    - Prediction inputs and outputs are automatically deleted after one hour -- persist outputs via webhooks or download immediately
    - The SDK auto-retries on 429 (rate limit) and 5xx errors -- 5 retries by default with exponential backoff. GET requests retry on 429 and 5xx; non-GET requests retry only on 429
    - `replicate.stream()` returns `ServerSentEvent` objects with `.event` (`"output"`, `"error"`, `"done"`) and `.data` (string) properties
    - File uploads are limited to 100 MiB -- for larger files, host them at a URL and pass the URL as input
    - Browser usage is not supported -- the SDK requires a server-side environment (Node.js 18+, Bun, Deno, Cloudflare Workers)
    - `webhook_events_filter` accepts `["start", "output", "logs", "completed"]` -- use `["completed"]` unless you need intermediate status updates
    - The `Prefer: wait` header enables sync mode on the HTTP API (up to 60s), but `replicate.run()` already handles this automatically
    - Community models may disappear or change without warning -- pin versions and maintain fallbacks for critical workflows
    - `replicate.wait()` polls the API until the prediction completes -- use webhooks for production to avoid polling overhead
    - `FileOutput.url()` returns the underlying URL, but these URLs are temporary -- download or persist the file before it expires
    
    </red_flags>
    
    ---
    
    <critical_reminders>
    
    ## CRITICAL REMINDERS
    
    > **All code must follow project conventions in CLAUDE.md** (kebab-case, named exports, import ordering, `import type`, named constants)
    
    **(You MUST never hardcode API tokens -- always use environment variables via `process.env.REPLICATE_API_TOKEN`)**
    
    **(You MUST handle `FileOutput` objects for models that return files -- do not assume outputs are plain strings or URLs)**
    
    **(You MUST validate webhooks using `validateWebhook()` from the `replicate` package -- never trust unverified webhook payloads)**
    
    **(You MUST account for cold starts when running infrequently-used models -- use deployments for latency-sensitive applications)**
    
    **(You MUST specify model versions (`owner/model:version`) in production to ensure reproducible results -- unversioned references use the latest, which can change)**
    
    **Failure to follow these rules will produce insecure, unreliable, or unpredictable AI integrations.**
    
    </critical_reminders>
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related