Claude Skill

ai-provider-cohere-sdk

Official Cohere TypeScript SDK patterns -- CohereClientV2, chat, embeddings, rerank, RAG with citations, tool use, streaming, and model selection

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Download agents-inc-skills-dist_plugins_ai-provider-cohere-sdk_skills_ai-provider-cohere-sdk-707467a.zip · 17 KB

Install

skills CLI npx skills add https://github.com/agents-inc/skills/tree/main/dist/plugins/ai-provider-cohere-sdk/skills/ai-provider-cohere-sdk
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install agents-inc-skills@llmmart
Git git clone https://github.com/agents-inc/skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole agents-inc/skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Cohere SDK Patterns

Quick Guide: Use the cohere-ai npm package with CohereClientV2 for all new Cohere integrations. V2 API requires model on every call. Use chatStream for streaming with content-delta events. Embeddings require inputType matching your use case (search_document for indexing, search_query for querying). Rerank scores documents by relevance. RAG works by passing documents to chat() -- the model returns inline citations automatically. Tool use follows a 4-step loop: user message, model returns tool_calls, you execute and return results, model generates cited response.


<critical_requirements>

CRITICAL: Before Using This Skill

(You MUST use CohereClientV2 (not CohereClient) for all new code -- V2 is the current API with required model parameter)

(You MUST specify inputType on every embed call -- search_document for indexing, search_query for querying -- mismatched types produce garbage similarity scores)

(You MUST handle the tool use loop correctly: append the full assistant message (with tool_calls) to messages, then append tool role results with matching tool_call_id)

(You MUST check finish_reason in responses -- MAX_TOKENS means the output was truncated)

(You MUST never hardcode API keys -- pass via token constructor parameter sourced from environment variables)

</critical_requirements>


Auto-detection: Cohere, cohere-ai, CohereClientV2, CohereClient, command-a, command-r, command-r-plus, embed-v4, rerank-v4, chatStream, content-delta, inputType, search_document, search_query, embeddingTypes, topN, CO_API_KEY, COHERE_API_KEY

When to use:

  • Building applications with Cohere Command models (chat, generation, summarization)
  • Creating semantic search pipelines with Cohere embeddings
  • Adding relevance scoring to search results with Cohere Rerank
  • Implementing RAG with inline document grounding and automatic citations
  • Building agentic workflows with Cohere tool use / function calling
  • Streaming chat responses for real-time user interfaces

Key patterns covered:

  • Client setup with CohereClientV2 (token, timeout, platform configs)
  • Chat and streaming (chat, chatStream, event types)
  • Embeddings with inputType for search/classification/clustering
  • Rerank for relevance scoring and search result ordering
  • RAG with documents and automatic citation handling
  • Tool use / function calling with multi-step loops
  • Model selection (Command-A, Command-R, Embed v4, Rerank v4)

When NOT to use:

  • Multi-provider applications needing OpenAI/Anthropic/Google switching -- use a unified provider SDK
  • React-specific chat UI hooks -- use a framework-integrated AI SDK
  • Simple text completion without Cohere-specific features (rerank, citations)

Examples Index





<decision_framework>

Decision Framework

Which Client Class to Use

New project?
+-- YES -> CohereClientV2 (always)
+-- Existing V1 code?
    +-- Working fine? -> Keep CohereClient but plan migration
    +-- Need V2 features? -> Migrate to CohereClientV2

Which Model to Choose

What is your task?
+-- General chat/generation -> command-a-03-2025 (most capable)
+-- Reasoning / multi-step -> command-a-reasoning-08-2025
+-- Image/document analysis -> command-a-vision-07-2025
+-- Translation -> command-a-translate-08-2025
+-- Lightweight / low latency -> command-r7b-12-2024
+-- Embeddings -> embed-v4.0 (or embed-english-v3.0 for English-only)
+-- Rerank quality -> rerank-v4.0-pro
+-- Rerank speed -> rerank-v4.0-fast

Embed inputType Selection

What are you embedding?
+-- Documents for a search index -> "search_document"
+-- Search queries against an index -> "search_query"
+-- Text for a classifier -> "classification"
+-- Text for clustering -> "clustering"
+-- Images -> "image" (embed-v4+ only)

When to Use Rerank

Do you have search results to re-order?
+-- YES -> Use rerank as a second-stage ranker
|   +-- Quality matters most? -> rerank-v4.0-pro
|   +-- Latency matters most? -> rerank-v4.0-fast
+-- NO -> Not applicable (rerank needs existing results to score)

RAG Approach

Do you need grounded answers with citations?
+-- YES -> Pass documents to chat()
|   +-- Have pre-retrieved documents? -> Pass directly via documents param
|   +-- Need retrieval first? -> Use embed + vector search + rerank pipeline, then pass top results to chat()
+-- NO -> Use plain chat without documents

</decision_framework>


<red_flags>

RED FLAGS

High Priority Issues:

  • Using CohereClient instead of CohereClientV2 for new code (V1 is legacy)
  • Missing model parameter in V2 API calls (required on every call, unlike V1)
  • Using wrong inputType for embeddings (search_query for documents or vice versa -- silently degrades results)
  • Hardcoding API keys instead of using environment variables
  • Not appending the full assistant message (with tool_calls) before appending tool results in the tool use loop

Medium Priority Issues:

  • Not specifying embeddingTypes (defaults may not match your storage format)
  • Ignoring finish_reason: "MAX_TOKENS" (output was silently truncated)
  • Not handling CohereTimeoutError separately from CohereError
  • Processing all stream events without checking type (only content-delta has text)
  • Using V1 parameter names (preamble, connectors, conversation_id) with V2 client

Common Mistakes:

  • Accessing response.text instead of response.message.content[0].text (V2 response shape changed)
  • Forgetting that embeddingTypes is required in V2 Embed API
  • Not matching tool_call_id when submitting tool results (model cannot correlate results)
  • Using documents with string values instead of { data: { text: "..." } } objects in V2
  • Expecting response.message.citations to exist when no documents were provided (citations only appear with grounded responses)

Gotchas & Edge Cases:

  • The SDK is in beta -- pin your cohere-ai version in package.json to avoid breaking changes
  • V2 API is NOT yet supported for cloud deployments (Bedrock, SageMaker, Azure, OCI) -- use V1 client for cloud platforms
  • inputType is camelCase in TypeScript SDK (inputType) but snake_case in the REST API (input_type)
  • Embed API accepts max 96 texts per call -- batch larger sets yourself
  • embed-v4.0 supports outputDimension for flexible sizing (256, 512, 1024, 1536) but v3 models have fixed dimensions
  • Rerank relevanceScore is normalized 0-1 but not calibrated across queries -- compare scores within a single query only
  • Stream events include tool-plan-delta before tool-call-start -- the model's reasoning about which tool to call
  • V2 uses system role for instructions (V1 used preamble parameter)
  • Citation sources in tool use responses reference tool_call_id values, not document indices
  • The clientName constructor parameter is for logging/analytics, not authentication
  • responseFormat: { type: "json_object" } is NOT supported in RAG mode (with documents, tools, or toolResults)
  • toolChoice is only supported on command-r7b-12-2024 and newer models
  • First requests with strictTools: true and a new tool set take longer (schema compilation)
  • thinking (reasoning mode) is only available on reasoning-capable models like command-a-reasoning-08-2025

</red_flags>


<critical_reminders>

CRITICAL REMINDERS

(You MUST use CohereClientV2 (not CohereClient) for all new code -- V2 is the current API with required model parameter)

(You MUST specify inputType on every embed call -- search_document for indexing, search_query for querying -- mismatched types produce garbage similarity scores)

(You MUST handle the tool use loop correctly: append the full assistant message (with tool_calls) to messages, then append tool role results with matching tool_call_id)

(You MUST check finish_reason in responses -- MAX_TOKENS means the output was truncated)

(You MUST never hardcode API keys -- pass via token constructor parameter sourced from environment variables)

Failure to follow these rules will produce broken embeddings, missing citations, or insecure AI integrations.

</critical_reminders>

Files (skills)
  • examples
    • core.md 5.9 KB
      Loading…
    • embeddings-rerank.md 6.6 KB
      Loading…
    • tools-rag.md 11.2 KB
      Loading…
  • reference.md 9.2 KB
    Loading…
  • SKILL.md 19.9 KB
    ---
    name: ai-provider-cohere-sdk
    description: Official Cohere TypeScript SDK patterns -- CohereClientV2, chat, embeddings, rerank, RAG with citations, tool use, streaming, and model selection
    ---
    
    # Cohere SDK Patterns
    
    > **Quick Guide:** Use the `cohere-ai` npm package with `CohereClientV2` for all new Cohere integrations. V2 API requires `model` on every call. Use `chatStream` for streaming with `content-delta` events. Embeddings require `inputType` matching your use case (`search_document` for indexing, `search_query` for querying). Rerank scores documents by relevance. RAG works by passing `documents` to `chat()` -- the model returns inline citations automatically. Tool use follows a 4-step loop: user message, model returns `tool_calls`, you execute and return results, model generates cited response.
    
    ---
    
    <critical_requirements>
    
    ## CRITICAL: Before Using This Skill
    
    **(You MUST use `CohereClientV2` (not `CohereClient`) for all new code -- V2 is the current API with required `model` parameter)**
    
    **(You MUST specify `inputType` on every embed call -- `search_document` for indexing, `search_query` for querying -- mismatched types produce garbage similarity scores)**
    
    **(You MUST handle the tool use loop correctly: append the full assistant message (with `tool_calls`) to messages, then append `tool` role results with matching `tool_call_id`)**
    
    **(You MUST check `finish_reason` in responses -- `MAX_TOKENS` means the output was truncated)**
    
    **(You MUST never hardcode API keys -- pass via `token` constructor parameter sourced from environment variables)**
    
    </critical_requirements>
    
    ---
    
    **Auto-detection:** Cohere, cohere-ai, CohereClientV2, CohereClient, command-a, command-r, command-r-plus, embed-v4, rerank-v4, chatStream, content-delta, inputType, search_document, search_query, embeddingTypes, topN, CO_API_KEY, COHERE_API_KEY
    
    **When to use:**
    
    - Building applications with Cohere Command models (chat, generation, summarization)
    - Creating semantic search pipelines with Cohere embeddings
    - Adding relevance scoring to search results with Cohere Rerank
    - Implementing RAG with inline document grounding and automatic citations
    - Building agentic workflows with Cohere tool use / function calling
    - Streaming chat responses for real-time user interfaces
    
    **Key patterns covered:**
    
    - Client setup with `CohereClientV2` (token, timeout, platform configs)
    - Chat and streaming (`chat`, `chatStream`, event types)
    - Embeddings with `inputType` for search/classification/clustering
    - Rerank for relevance scoring and search result ordering
    - RAG with documents and automatic citation handling
    - Tool use / function calling with multi-step loops
    - Model selection (Command-A, Command-R, Embed v4, Rerank v4)
    
    **When NOT to use:**
    
    - Multi-provider applications needing OpenAI/Anthropic/Google switching -- use a unified provider SDK
    - React-specific chat UI hooks -- use a framework-integrated AI SDK
    - Simple text completion without Cohere-specific features (rerank, citations)
    
    ---
    
    ## Examples Index
    
    - [Core: Setup, Chat & Error Handling](examples/core.md) -- CohereClientV2 init, basic chat, streaming, error handling
    - [Embeddings & Rerank](examples/embeddings-rerank.md) -- Semantic search, input types, rerank scoring, RAG pipeline
    - [Tool Use & RAG](examples/tools-rag.md) -- Function calling, document grounding, citation handling
    - [Quick API Reference](reference.md) -- Model IDs, method signatures, event types, error classes
    
    ---
    
    <philosophy>
    
    ## Philosophy
    
    The Cohere TypeScript SDK (`cohere-ai`) provides **direct access to Cohere's API surface** -- chat, embeddings, rerank, and RAG with citations. The SDK is auto-generated from Cohere's API spec using Fern.
    
    **Core principles:**
    
    1. **V2 API is current** -- `CohereClientV2` provides the modern API. `model` is required on every call. V1 methods on `CohereClient` are legacy.
    2. **Embeddings are typed** -- The `inputType` parameter (`search_document`, `search_query`, `classification`, `clustering`) is mandatory for v3+ models. Mismatching input types between indexing and querying silently degrades results.
    3. **RAG is first-class** -- Pass `documents` directly to `chat()` and the model returns grounded answers with inline citations. No external retrieval framework required for the grounding step.
    4. **Rerank is a standalone primitive** -- Score and reorder search results without building a full RAG pipeline. Feed any list of documents and a query, get relevance scores back.
    5. **Citations are automatic** -- When documents are provided (via RAG or tool results), the model generates fine-grained citations with start/end positions and source references.
    
    **When to use the Cohere SDK directly:**
    
    - You want Cohere-specific features: rerank, citation grounding, multilingual embeddings
    - You need semantic search with embed + rerank pipeline
    - You want RAG with automatic inline citations
    - You are building on Cohere's platform (or Bedrock/Azure/OCI with Cohere models)
    
    **When NOT to use:**
    
    - You need to switch between multiple LLM providers -- use a unified provider SDK
    - You want React-specific chat UI hooks -- use a framework-integrated AI SDK
    - You only need basic chat completion without Cohere differentiators
    
    </philosophy>
    
    ---
    
    <patterns>
    
    ## Core Patterns
    
    ### Pattern 1: Client Setup
    
    Initialize `CohereClientV2`. The `token` parameter is required (pass from environment).
    
    ```typescript
    // lib/cohere.ts -- basic setup
    import { CohereClientV2 } from "cohere-ai";
    
    const client = new CohereClientV2({
      token: process.env.CO_API_KEY,
    });
    
    export { client };
    ```
    
    ```typescript
    // lib/cohere.ts -- production configuration
    const TIMEOUT_MS = 30_000;
    
    const client = new CohereClientV2({
      token: process.env.CO_API_KEY,
      timeout: TIMEOUT_MS,
    });
    ```
    
    **Why good:** Explicit token from env var, named timeout constant, named export
    
    ```typescript
    // BAD: Hardcoded key, default CohereClient (V1)
    import { CohereClient } from "cohere-ai";
    const client = new CohereClient({ token: "sk-abc123" });
    ```
    
    **Why bad:** Hardcoded API key is a security breach risk, `CohereClient` is the legacy V1 client
    
    **See:** [examples/core.md](examples/core.md) for error handling, platform configs (Bedrock, Azure)
    
    ---
    
    ### Pattern 2: Chat Completion
    
    V2 chat uses `messages` array with `system`, `user`, `assistant`, and `tool` roles.
    
    ```typescript
    const response = await client.chat({
      model: "command-a-03-2025",
      messages: [
        { role: "system", content: "You are a helpful coding assistant." },
        { role: "user", content: "Explain TypeScript generics." },
      ],
    });
    
    console.log(response.message.content[0].text);
    ```
    
    **Why good:** System message for instruction, `model` explicitly specified, correct V2 content access path
    
    ```typescript
    // BAD: Missing model (required in V2), wrong response access
    const response = await client.chat({
      messages: [{ role: "user", content: "Hello" }],
    });
    console.log(response.text); // WRONG: V2 uses response.message.content[0].text
    ```
    
    **Why bad:** V2 requires `model`, response shape is `response.message.content[0].text` not `response.text`
    
    **See:** [examples/core.md](examples/core.md) for multi-turn, token tracking, temperature control
    
    ---
    
    ### Pattern 3: Streaming
    
    Use `chatStream` with `for await` and check event `type` for `content-delta`.
    
    ```typescript
    const stream = await client.chatStream({
      model: "command-a-03-2025",
      messages: [{ role: "user", content: "Explain async/await." }],
    });
    
    for await (const event of stream) {
      if (event.type === "content-delta") {
        process.stdout.write(event.delta?.message?.content?.text ?? "");
      }
    }
    ```
    
    **Why good:** Checks event type before accessing delta, handles nullable content safely
    
    ```typescript
    // BAD: Not checking event type
    for await (const event of stream) {
      console.log(event.delta?.message); // Many events don't have message delta
    }
    ```
    
    **Why bad:** Only `content-delta` events have text content -- other events (`message-start`, `citation-start`, `tool-plan-delta`) have different shapes
    
    **See:** [examples/core.md](examples/core.md) for full streaming with all event types
    
    ---
    
    ### Pattern 4: Embeddings
    
    `inputType` is required for v3+ models. Mismatching types between indexing and querying silently degrades results.
    
    ```typescript
    const EMBEDDING_MODEL = "embed-v4.0";
    
    // Index documents with search_document
    const docEmbeddings = await client.embed({
      model: EMBEDDING_MODEL,
      inputType: "search_document",
      texts: ["TypeScript is a typed superset of JavaScript."],
      embeddingTypes: ["float"],
    });
    
    // Query with search_query
    const queryEmbedding = await client.embed({
      model: EMBEDDING_MODEL,
      inputType: "search_query",
      texts: ["What is TypeScript?"],
      embeddingTypes: ["float"],
    });
    ```
    
    **Why good:** Correct `inputType` pairing, `embeddingTypes` explicitly specified, named model constant
    
    ```typescript
    // BAD: Same inputType for both indexing and querying
    const docs = await client.embed({
      model: "embed-v4.0",
      inputType: "search_query", // WRONG for documents
      texts: documents,
      embeddingTypes: ["float"],
    });
    ```
    
    **Why bad:** Using `search_query` for document indexing silently produces worse similarity scores -- documents must use `search_document`
    
    **See:** [examples/embeddings-rerank.md](examples/embeddings-rerank.md) for cosine similarity, dimension control, batch embedding
    
    ---
    
    ### Pattern 5: Rerank
    
    Score documents by relevance to a query. Returns ordered results with relevance scores.
    
    ```typescript
    const RERANK_MODEL = "rerank-v4.0-pro";
    const TOP_N = 3;
    
    const result = await client.rerank({
      model: RERANK_MODEL,
      query: "What is TypeScript?",
      documents: [
        "TypeScript is a typed superset of JavaScript.",
        "Python is a general-purpose language.",
        "TypeScript compiles to JavaScript.",
      ],
      topN: TOP_N,
    });
    
    for (const item of result.results) {
      console.log(`Doc ${item.index}: score ${item.relevanceScore}`);
    }
    ```
    
    **Why good:** Named constants, `topN` limits results, accesses `index` and `relevanceScore`
    
    **See:** [examples/embeddings-rerank.md](examples/embeddings-rerank.md) for embed + rerank pipeline, rank fields
    
    ---
    
    ### Pattern 6: RAG with Documents
    
    Pass `documents` to `chat()` and the model returns grounded answers with inline citations.
    
    ```typescript
    const response = await client.chat({
      model: "command-a-03-2025",
      messages: [{ role: "user", content: "What is TypeScript?" }],
      documents: [
        {
          data: {
            text: "TypeScript is a typed superset of JavaScript.",
            title: "TS Docs",
          },
        },
        {
          data: {
            text: "TypeScript was developed by Microsoft.",
            title: "History",
          },
        },
      ],
    });
    
    console.log(response.message.content[0].text);
    
    // Citations reference which documents support each claim
    if (response.message.citations) {
      for (const citation of response.message.citations) {
        console.log(`"${citation.text}" from doc ${citation.sources}`);
      }
    }
    ```
    
    **Why good:** Documents passed inline with metadata, citations accessed from response, no external retrieval framework needed
    
    **See:** [examples/tools-rag.md](examples/tools-rag.md) for full RAG pipeline with embed + rerank + chat
    
    ---
    
    ### Pattern 7: Tool Use / Function Calling
    
    4-step loop: user message -> model returns `tool_calls` -> execute tools -> return results with `tool_call_id`.
    
    ```typescript
    const tools = [
      {
        type: "function" as const,
        function: {
          name: "get_weather",
          description: "Get weather for a city",
          parameters: {
            type: "object",
            properties: {
              location: { type: "string", description: "City name" },
            },
            required: ["location"],
          },
        },
      },
    ];
    
    const response = await client.chat({
      model: "command-a-03-2025",
      messages: [{ role: "user", content: "Weather in Paris?" }],
      tools,
    });
    
    // Check if model wants to call tools
    if (response.message.toolCalls) {
      // See examples/tools-rag.md for the complete tool execution loop
    }
    ```
    
    **Why good:** Standard JSON Schema tool definition, checks for toolCalls before executing
    
    **See:** [examples/tools-rag.md](examples/tools-rag.md) for complete multi-step tool loop with tool result submission
    
    ---
    
    ### Pattern 8: Error Handling
    
    Catch `CohereError` for API errors, `CohereTimeoutError` for timeouts.
    
    ```typescript
    import { CohereError, CohereTimeoutError } from "cohere-ai";
    
    try {
      const response = await client.chat({
        model: "command-a-03-2025",
        messages: [{ role: "user", content: "Hello" }],
      });
    } catch (error) {
      if (error instanceof CohereTimeoutError) {
        console.error("Request timed out");
      } else if (error instanceof CohereError) {
        console.error(`API Error [${error.statusCode}]: ${error.message}`);
        console.error("Body:", error.body);
      } else {
        throw error; // Re-throw unknown errors
      }
    }
    ```
    
    **Why good:** Specific error types with status codes, re-throws unexpected errors, timeout handled separately
    
    **See:** [examples/core.md](examples/core.md) for production error handling patterns
    
    </patterns>
    
    ---
    
    <performance>
    
    ## Performance Optimization
    
    ### Model Selection for Cost/Speed
    
    ```
    General purpose (best)      -> command-a-03-2025 (256K context, strongest)
    Reasoning tasks             -> command-a-reasoning-08-2025 (multi-step reasoning)
    Vision/document analysis    -> command-a-vision-07-2025 (images, charts, OCR)
    Translation                 -> command-a-translate-08-2025 (23 languages)
    Lightweight / edge          -> command-r7b-12-2024 (7B, fast, 128K context)
    Legacy (still supported)    -> command-r-08-2024, command-r-plus-08-2024
    Embeddings (best)           -> embed-v4.0 (multimodal, 128K context, flexible dims)
    Embeddings (English)        -> embed-english-v3.0 (1024 dims)
    Embeddings (multilingual)   -> embed-multilingual-v3.0 (23 languages)
    Rerank (quality)            -> rerank-v4.0-pro (32K context, multilingual)
    Rerank (speed)              -> rerank-v4.0-fast (32K context, latency-optimized)
    ```
    
    ### Key Optimization Patterns
    
    - **Batch embeddings** -- pass up to 96 texts per `embed()` call instead of calling per-document
    - **Use `topN` in rerank** -- limit results to reduce response size and cost
    - **Use `outputDimension` with embed-v4** -- reduce dimensions (256/512/1024) for faster similarity search at minimal quality loss
    - **Check `finish_reason === "MAX_TOKENS"`** -- detect truncated output
    - **Use `temperature: 0`** for deterministic output (enables caching)
    - **Use embed-v4 `int8`/`binary` types** for compressed storage with minimal quality loss
    - **Use `strictTools: true`** to force tool calls to follow the schema exactly (structured outputs)
    - **Use `thinking: { type: "enabled" }`** with reasoning models for complex multi-step tasks
    - **Use `toolChoice: "REQUIRED"`** when you always want the model to call a tool (command-r7b+ only)
    
    </performance>
    
    ---
    
    <decision_framework>
    
    ## Decision Framework
    
    ### Which Client Class to Use
    
    ```
    New project?
    +-- YES -> CohereClientV2 (always)
    +-- Existing V1 code?
        +-- Working fine? -> Keep CohereClient but plan migration
        +-- Need V2 features? -> Migrate to CohereClientV2
    ```
    
    ### Which Model to Choose
    
    ```
    What is your task?
    +-- General chat/generation -> command-a-03-2025 (most capable)
    +-- Reasoning / multi-step -> command-a-reasoning-08-2025
    +-- Image/document analysis -> command-a-vision-07-2025
    +-- Translation -> command-a-translate-08-2025
    +-- Lightweight / low latency -> command-r7b-12-2024
    +-- Embeddings -> embed-v4.0 (or embed-english-v3.0 for English-only)
    +-- Rerank quality -> rerank-v4.0-pro
    +-- Rerank speed -> rerank-v4.0-fast
    ```
    
    ### Embed `inputType` Selection
    
    ```
    What are you embedding?
    +-- Documents for a search index -> "search_document"
    +-- Search queries against an index -> "search_query"
    +-- Text for a classifier -> "classification"
    +-- Text for clustering -> "clustering"
    +-- Images -> "image" (embed-v4+ only)
    ```
    
    ### When to Use Rerank
    
    ```
    Do you have search results to re-order?
    +-- YES -> Use rerank as a second-stage ranker
    |   +-- Quality matters most? -> rerank-v4.0-pro
    |   +-- Latency matters most? -> rerank-v4.0-fast
    +-- NO -> Not applicable (rerank needs existing results to score)
    ```
    
    ### RAG Approach
    
    ```
    Do you need grounded answers with citations?
    +-- YES -> Pass documents to chat()
    |   +-- Have pre-retrieved documents? -> Pass directly via documents param
    |   +-- Need retrieval first? -> Use embed + vector search + rerank pipeline, then pass top results to chat()
    +-- NO -> Use plain chat without documents
    ```
    
    </decision_framework>
    
    ---
    
    <red_flags>
    
    ## RED FLAGS
    
    **High Priority Issues:**
    
    - Using `CohereClient` instead of `CohereClientV2` for new code (V1 is legacy)
    - Missing `model` parameter in V2 API calls (required on every call, unlike V1)
    - Using wrong `inputType` for embeddings (`search_query` for documents or vice versa -- silently degrades results)
    - Hardcoding API keys instead of using environment variables
    - Not appending the full assistant message (with `tool_calls`) before appending tool results in the tool use loop
    
    **Medium Priority Issues:**
    
    - Not specifying `embeddingTypes` (defaults may not match your storage format)
    - Ignoring `finish_reason: "MAX_TOKENS"` (output was silently truncated)
    - Not handling `CohereTimeoutError` separately from `CohereError`
    - Processing all stream events without checking `type` (only `content-delta` has text)
    - Using V1 parameter names (`preamble`, `connectors`, `conversation_id`) with V2 client
    
    **Common Mistakes:**
    
    - Accessing `response.text` instead of `response.message.content[0].text` (V2 response shape changed)
    - Forgetting that `embeddingTypes` is required in V2 Embed API
    - Not matching `tool_call_id` when submitting tool results (model cannot correlate results)
    - Using `documents` with string values instead of `{ data: { text: "..." } }` objects in V2
    - Expecting `response.message.citations` to exist when no documents were provided (citations only appear with grounded responses)
    
    **Gotchas & Edge Cases:**
    
    - The SDK is in beta -- pin your `cohere-ai` version in package.json to avoid breaking changes
    - V2 API is NOT yet supported for cloud deployments (Bedrock, SageMaker, Azure, OCI) -- use V1 client for cloud platforms
    - `inputType` is camelCase in TypeScript SDK (`inputType`) but snake_case in the REST API (`input_type`)
    - Embed API accepts max 96 texts per call -- batch larger sets yourself
    - `embed-v4.0` supports `outputDimension` for flexible sizing (256, 512, 1024, 1536) but v3 models have fixed dimensions
    - Rerank `relevanceScore` is normalized 0-1 but not calibrated across queries -- compare scores within a single query only
    - Stream events include `tool-plan-delta` before `tool-call-start` -- the model's reasoning about which tool to call
    - V2 uses `system` role for instructions (V1 used `preamble` parameter)
    - Citation `sources` in tool use responses reference `tool_call_id` values, not document indices
    - The `clientName` constructor parameter is for logging/analytics, not authentication
    - `responseFormat: { type: "json_object" }` is NOT supported in RAG mode (with `documents`, `tools`, or `toolResults`)
    - `toolChoice` is only supported on `command-r7b-12-2024` and newer models
    - First requests with `strictTools: true` and a new tool set take longer (schema compilation)
    - `thinking` (reasoning mode) is only available on reasoning-capable models like `command-a-reasoning-08-2025`
    
    </red_flags>
    
    ---
    
    <critical_reminders>
    
    ## CRITICAL REMINDERS
    
    **(You MUST use `CohereClientV2` (not `CohereClient`) for all new code -- V2 is the current API with required `model` parameter)**
    
    **(You MUST specify `inputType` on every embed call -- `search_document` for indexing, `search_query` for querying -- mismatched types produce garbage similarity scores)**
    
    **(You MUST handle the tool use loop correctly: append the full assistant message (with `tool_calls`) to messages, then append `tool` role results with matching `tool_call_id`)**
    
    **(You MUST check `finish_reason` in responses -- `MAX_TOKENS` means the output was truncated)**
    
    **(You MUST never hardcode API keys -- pass via `token` constructor parameter sourced from environment variables)**
    
    **Failure to follow these rules will produce broken embeddings, missing citations, or insecure AI integrations.**
    
    </critical_reminders>
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.