# Together AI SDK -- Chat Completions Examples

> Chat Completions API patterns: basic completion, multi-turn conversations, model selection, vision, and token control. See [SKILL.md](../SKILL.md) for core patterns.

**Related examples:**

- [core.md](core.md) -- Client setup, error handling
- [streaming.md](streaming.md) -- Streaming responses
- [tools.md](tools.md) -- Tool/function calling
- [structured-output.md](structured-output.md) -- Structured outputs with Zod
- [images.md](images.md) -- Image generation, embeddings

---

## Basic Chat Completion

```typescript
// basic-chat.ts
import Together from "together-ai";

const client = new Together();

async function chat(userMessage: string): Promise<string> {
  const completion = await client.chat.completions.create({
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages: [
      {
        role: "system",
        content: "You are a helpful assistant. Be concise.",
      },
      { role: "user", content: userMessage },
    ],
  });

  const content = completion.choices[0].message.content;
  if (!content) {
    throw new Error("No content in response");
  }

  return content;
}

const answer = await chat("What is TypeScript in one sentence?");
console.log(answer);
```

---

## Multi-Turn Conversations

```typescript
import Together from "together-ai";
import type { Together as TogetherTypes } from "together-ai";

const client = new Together();

const messages: TogetherTypes.Chat.CompletionCreateParams["messages"] = [
  { role: "system", content: "You are a TypeScript expert." },
  { role: "user", content: "What is a union type?" },
];

const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages,
});

// Append assistant response for next turn
const assistantMessage = completion.choices[0].message;
messages.push({ role: "assistant", content: assistantMessage.content ?? "" });
messages.push({ role: "user", content: "Give me a real-world example." });

const followUp = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages,
});
```

---

## Controlling Output Length and Temperature

```typescript
const MAX_TOKENS = 500;

const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Summarize this article." }],
  max_tokens: MAX_TOKENS,
  temperature: 0, // Deterministic output
});

const finishReason = completion.choices[0].finish_reason;
if (finishReason === "length") {
  console.warn("Output was truncated -- increase max_tokens");
}
```

---

## Vision -- Image from URL

```typescript
import Together from "together-ai";

const client = new Together();

async function analyzeImage(
  imageUrl: string,
  question: string,
): Promise<string> {
  const response = await client.chat.completions.create({
    model: "Qwen/Qwen3-VL-8B-Instruct",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: question },
          { type: "image_url", image_url: { url: imageUrl } },
        ],
      },
    ],
  });

  return response.choices[0].message.content ?? "";
}

const description = await analyzeImage(
  "https://example.com/photo.jpg",
  "Describe what you see in this image.",
);
console.log(description);
```

---

## Vision -- Multiple Images

```typescript
const response = await client.chat.completions.create({
  model: "Qwen/Qwen3-VL-8B-Instruct",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Compare these two images." },
        {
          type: "image_url",
          image_url: { url: "https://example.com/image1.jpg" },
        },
        {
          type: "image_url",
          image_url: { url: "https://example.com/image2.jpg" },
        },
      ],
    },
  ],
});
```

---

_For core concepts, see [SKILL.md](../SKILL.md). For API reference tables, see [reference.md](../reference.md)._

# Together AI SDK -- Setup & Configuration Examples

> Client initialization, environment config, production settings, error handling, and OpenAI compatibility. See [SKILL.md](../SKILL.md) for core patterns.

**Related examples:**

- [chat.md](chat.md) -- Chat Completions API
- [streaming.md](streaming.md) -- Streaming responses
- [tools.md](tools.md) -- Tool/function calling
- [structured-output.md](structured-output.md) -- Structured outputs with Zod
- [images.md](images.md) -- Image generation, embeddings

---

## Basic Client Setup

```typescript
// lib/together.ts
import Together from "together-ai";

// Reads TOGETHER_API_KEY from env automatically
const client = new Together();

export { client };
```

---

## Production Configuration

```typescript
// lib/together.ts
import Together from "together-ai";

const TIMEOUT_MS = 30_000;
const MAX_RETRIES = 3;

const client = new Together({
  apiKey: process.env.TOGETHER_API_KEY,
  timeout: TIMEOUT_MS,
  maxRetries: MAX_RETRIES,
});

export { client };
```

---

## Production Error Handling

```typescript
// error-handling.ts
import Together from "together-ai";

const TIMEOUT_MS = 30_000;
const MAX_RETRIES = 3;

const client = new Together({
  timeout: TIMEOUT_MS,
  maxRetries: MAX_RETRIES,
});

async function safeCompletion(prompt: string): Promise<string | null> {
  try {
    const completion = await client.chat.completions.create({
      model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: prompt },
      ],
    });

    const content = completion.choices[0].message.content;
    if (!content) {
      throw new Error("No content in response");
    }

    return content;
  } catch (error) {
    if (error instanceof Together.APIError) {
      console.error(`Together API Error [${error.status}]: ${error.message}`);

      if (error instanceof Together.RateLimitError) {
        console.error("Rate limited. SDK will auto-retry.");
        // If we get here, all retries were exhausted
        return null;
      }

      if (error instanceof Together.AuthenticationError) {
        throw new Error(
          "Invalid API key. Check TOGETHER_API_KEY environment variable.",
        );
      }

      if (error instanceof Together.BadRequestError) {
        console.error("Invalid request parameters:", error.message);
        return null;
      }

      if (error instanceof Together.InternalServerError) {
        console.error("Together server error after all retries");
        return null;
      }
    }

    // Network/connection errors
    if (error instanceof Together.APIConnectionError) {
      console.error("Network error:", error.message);
      return null;
    }

    // Unknown errors should be re-thrown
    throw error;
  }
}

const result = await safeCompletion("Hello!");
if (result) {
  console.log(result);
} else {
  console.error("Failed to get completion");
}
```

---

## Error Type Hierarchy

```typescript
// Error class hierarchy:
// Together.APIError (base)
//   +-- Together.BadRequestError          (400)
//   +-- Together.AuthenticationError      (401)
//   +-- Together.PermissionDeniedError    (403)
//   +-- Together.NotFoundError            (404)
//   +-- Together.UnprocessableEntityError (422)
//   +-- Together.RateLimitError           (429)
//   +-- Together.InternalServerError      (>=500)
//   +-- Together.APIConnectionError       (network)
```

---

## OpenAI SDK Compatibility

Use the OpenAI SDK with Together AI by swapping the base URL.

```typescript
// lib/together-openai.ts
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOGETHER_API_KEY,
  baseURL: "https://api.together.xyz/v1",
});

// Use exactly like OpenAI SDK, but with Together model IDs
const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello!" },
  ],
});

console.log(completion.choices[0].message.content);
export { client };
```

---

## Per-Request Overrides

```typescript
// Override retries and timeout for a single request
await client.chat.completions.create(
  {
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages: [{ role: "user", content: "Hello" }],
  },
  {
    maxRetries: 5,
    timeout: 60_000,
    signal: abortController.signal,
    headers: { "X-Custom-Header": "value" },
  },
);
```

---

## Request Cancellation with AbortController

```typescript
const controller = new AbortController();
const ABORT_TIMEOUT_MS = 5_000;

// Cancel after timeout
setTimeout(() => controller.abort(), ABORT_TIMEOUT_MS);

try {
  const completion = await client.chat.completions.create(
    {
      model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
      messages: [{ role: "user", content: "Hello" }],
    },
    { signal: controller.signal },
  );
} catch (error) {
  if (error instanceof Error && error.name === "AbortError") {
    console.log("Request was cancelled");
  }
}
```

---

## Raw Response Access

```typescript
// Get underlying Response object
const response = await client.chat.completions
  .create({
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages: [{ role: "user", content: "Hello" }],
  })
  .asResponse();

console.log(response.status);
console.log(response.headers.get("x-request-id"));

// Or get both data and response
const { data, response: raw } = await client.chat.completions
  .create({
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages: [{ role: "user", content: "Hello" }],
  })
  .withResponse();
```

---

_For core concepts, see [SKILL.md](../SKILL.md). For API reference tables, see [reference.md](../reference.md)._

# Together AI SDK -- Images & Embeddings Examples

> Image generation with FLUX/Stable Diffusion and embeddings for semantic search. See [SKILL.md](../SKILL.md) for core patterns.

**Related examples:**

- [core.md](core.md) -- Client setup, error handling
- [chat.md](chat.md) -- Chat Completions API
- [streaming.md](streaming.md) -- Streaming responses
- [tools.md](tools.md) -- Tool/function calling
- [structured-output.md](structured-output.md) -- Structured outputs with Zod

---

## Basic Image Generation (FLUX schnell)

```typescript
// image-generation.ts
import Together from "together-ai";

const client = new Together();

const response = await client.images.generate({
  model: "black-forest-labs/FLUX.1-schnell",
  prompt: "A serene mountain landscape at sunset with a lake reflection",
  steps: 4,
});

console.log(response.data[0].url);
```

---

## High Quality Image (FLUX Pro)

```typescript
import Together from "together-ai";

const client = new Together();

const IMAGE_WIDTH = 1024;
const IMAGE_HEIGHT = 768;
const IMAGE_STEPS = 25;

const response = await client.images.generate({
  model: "black-forest-labs/FLUX.1.1-pro",
  prompt: "A photorealistic portrait of a robot in a garden",
  width: IMAGE_WIDTH,
  height: IMAGE_HEIGHT,
  steps: IMAGE_STEPS,
});

console.log(response.data[0].url);
```

---

## Multiple Image Variations

```typescript
import Together from "together-ai";

const client = new Together();

const NUM_VARIATIONS = 4;

const response = await client.images.generate({
  model: "black-forest-labs/FLUX.1-schnell",
  prompt: "A cute robot assistant helping in a modern office",
  n: NUM_VARIATIONS,
  steps: 4,
});

response.data.forEach((image, index) => {
  console.log(`Variation ${index + 1}: ${image.url}`);
});
```

---

## Base64 Response Format

```typescript
import Together from "together-ai";

const client = new Together();

const response = await client.images.generate({
  model: "black-forest-labs/FLUX.1-schnell",
  prompt: "A cat in outer space",
  response_format: "base64",
});

const base64Data = response.data[0].b64_json;
// Use directly in <img src="data:image/png;base64,${base64Data}" />
```

---

## Image Editing with Reference Images (FLUX.2)

```typescript
import Together from "together-ai";

const client = new Together();

const response = await client.images.generate({
  model: "black-forest-labs/FLUX.2-pro",
  prompt: "Replace the color of the car to blue",
  width: 1024,
  height: 768,
  reference_images: ["https://example.com/original-car.jpg"],
});

console.log(response.data[0].url);
```

---

## Kontext Image Editing

```typescript
import Together from "together-ai";

const client = new Together();

const response = await client.images.generate({
  model: "black-forest-labs/FLUX.1-kontext-pro",
  prompt: "Add a party hat to the dog",
  image_url: "https://example.com/dog.jpg",
});

console.log(response.data[0].url);
```

---

## Image Model Selection Guide

```
Fast generation (4 steps)      -> FLUX.1 schnell (aspect_ratio param)
High quality                   -> FLUX.1.1 pro (width/height params)
Latest + reference images      -> FLUX.2 pro (width/height + reference_images)
Image editing (single ref)     -> FLUX.1 Kontext pro (aspect_ratio + image_url)
Negative prompts               -> Stable Diffusion models (negative_prompt param)
```

---

## Basic Embeddings

```typescript
// embeddings.ts
import Together from "together-ai";

const client = new Together();
const EMBEDDING_MODEL = "BAAI/bge-large-en-v1.5";

const response = await client.embeddings.create({
  model: EMBEDDING_MODEL,
  input: "TypeScript provides static type checking for JavaScript.",
});

console.log("Embedding dimensions:", response.data[0].embedding.length);
console.log("First 5 values:", response.data[0].embedding.slice(0, 5));
```

---

## Batch Embeddings

```typescript
import Together from "together-ai";

const client = new Together();
const EMBEDDING_MODEL = "BAAI/bge-large-en-v1.5";

const documents = [
  "TypeScript provides static type checking.",
  "React is a library for building user interfaces.",
  "Node.js is a JavaScript runtime built on V8.",
  "PostgreSQL is a powerful relational database.",
];

// Batch all inputs in one call for efficiency
const response = await client.embeddings.create({
  model: EMBEDDING_MODEL,
  input: documents,
});

const embeddings = response.data.map((item) => ({
  index: item.index,
  embedding: item.embedding,
}));

console.log(`Generated ${embeddings.length} embeddings`);
```

---

## Semantic Search with Cosine Similarity

```typescript
import Together from "together-ai";

const client = new Together();
const EMBEDDING_MODEL = "BAAI/bge-large-en-v1.5";
const SIMILARITY_THRESHOLD = 0.7;
const TOP_K = 3;

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0;
  let normA = 0;
  let normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

// Index documents
const documents = [
  "TypeScript provides static type checking for JavaScript.",
  "React is a library for building user interfaces.",
  "PostgreSQL is a powerful relational database.",
  "Docker containers package applications with dependencies.",
];

const docEmbeddings = await client.embeddings.create({
  model: EMBEDDING_MODEL,
  input: documents,
});

const indexedDocs = documents.map((text, i) => ({
  text,
  embedding: docEmbeddings.data[i].embedding,
}));

// Search
async function search(
  query: string,
): Promise<Array<{ text: string; score: number }>> {
  const queryEmbedding = await client.embeddings.create({
    model: EMBEDDING_MODEL,
    input: query,
  });

  const queryVector = queryEmbedding.data[0].embedding;

  return indexedDocs
    .map((doc) => ({
      text: doc.text,
      score: cosineSimilarity(queryVector, doc.embedding),
    }))
    .filter((r) => r.score > SIMILARITY_THRESHOLD)
    .sort((a, b) => b.score - a.score)
    .slice(0, TOP_K);
}

const results = await search("What is TypeScript?");
results.forEach((r) => {
  console.log(`[${r.score.toFixed(3)}] ${r.text}`);
});
```

---

## Fine-Tuning (File Upload + Job Creation)

```typescript
import Together from "together-ai";
import { createReadStream } from "node:fs";

const client = new Together();
const EPOCHS = 3;

// Upload training data (JSONL format)
const file = await client.files.upload({
  file: createReadStream("training-data.jsonl"),
  purpose: "fine-tune",
});
console.log(`Uploaded file: ${file.id}`);

// Create fine-tuning job
const job = await client.fineTuning.create({
  training_file: file.id,
  model: "meta-llama/Meta-Llama-3-8B-Instruct",
  n_epochs: EPOCHS,
});
console.log(`Fine-tuning job: ${job.id}`);

// Monitor job status
const status = await client.fineTuning.retrieve(job.id);
console.log(`Status: ${status.status}`);

// List events
const events = await client.fineTuning.listEvents(job.id);
console.log("Events:", events);
```

### Training Data Format (JSONL)

Each line is a JSON object with a `messages` array:

```jsonl
{"messages": [{"role": "system", "content": "You are helpful."}, {"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}
{"messages": [{"role": "user", "content": "What is TypeScript?"}, {"role": "assistant", "content": "TypeScript is a typed superset of JavaScript."}]}
```

---

_For core concepts, see [SKILL.md](../SKILL.md). For API reference tables, see [reference.md](../reference.md)._

# Together AI SDK -- Streaming Examples

> Streaming patterns: `stream: true` with async iterators, stream cancellation, and controller access. See [SKILL.md](../SKILL.md) for core patterns.

**Related examples:**

- [core.md](core.md) -- Client setup, error handling
- [chat.md](chat.md) -- Chat Completions API
- [tools.md](tools.md) -- Tool/function calling
- [structured-output.md](structured-output.md) -- Structured outputs with Zod
- [images.md](images.md) -- Image generation, embeddings

---

## Basic Streaming with `for await`

```typescript
// streaming-chat.ts
import Together from "together-ai";

const client = new Together();

const stream = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Explain async/await in TypeScript." },
  ],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
  }
}
console.log(); // newline
```

---

## Collecting Full Response from Stream

```typescript
import Together from "together-ai";

const client = new Together();

async function streamToString(prompt: string): Promise<string> {
  const stream = await client.chat.completions.create({
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: prompt },
    ],
    stream: true,
  });

  const parts: string[] = [];

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) {
      parts.push(content);
      process.stdout.write(content); // Show progress
    }
  }
  console.log(); // newline

  return parts.join("");
}

const result = await streamToString("Explain promises in JavaScript.");
console.log("Total length:", result.length);
```

---

## Stream Cancellation

```typescript
import Together from "together-ai";

const client = new Together();

const stream = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Tell me a long story." }],
  stream: true,
});

let tokenCount = 0;
const MAX_TOKENS_TO_READ = 100;

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) {
    process.stdout.write(content);
    tokenCount++;
    if (tokenCount >= MAX_TOKENS_TO_READ) {
      // Cancel the stream by using the controller
      stream.controller.abort();
      break;
    }
  }
}
console.log("\nStream cancelled after", tokenCount, "chunks");
```

---

## Streaming with Tool Calls

```typescript
import Together from "together-ai";

const client = new Together();

const stream = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "What is the weather in Tokyo?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get weather for a city",
        parameters: {
          type: "object",
          properties: {
            location: { type: "string" },
          },
          required: ["location"],
        },
      },
    },
  ],
  stream: true,
});

for await (const chunk of stream) {
  const toolCalls = chunk.choices[0]?.delta?.tool_calls ?? [];
  for (const toolCall of toolCalls) {
    console.log("Tool call delta:", toolCall);
  }
}
```

---

_For core concepts, see [SKILL.md](../SKILL.md). For API reference tables, see [reference.md](../reference.md)._

# Together AI SDK -- Structured Output Examples

> Structured output patterns: JSON schema mode with Zod, regex mode, vision with JSON, and complex schemas. See [SKILL.md](../SKILL.md) for core patterns.

**Related examples:**

- [core.md](core.md) -- Client setup, error handling
- [chat.md](chat.md) -- Chat Completions API
- [streaming.md](streaming.md) -- Streaming responses
- [tools.md](tools.md) -- Tool/function calling
- [images.md](images.md) -- Image generation, embeddings

---

## JSON Schema Mode with Zod

```typescript
// structured-output.ts
import Together from "together-ai";
import { z } from "zod";

const client = new Together();

const VoiceNoteSchema = z.object({
  title: z.string().describe("A title for the voice note"),
  summary: z.string().describe("A one sentence summary"),
  actionItems: z
    .array(z.string())
    .describe("A list of action items from the note"),
});

type VoiceNote = z.infer<typeof VoiceNoteSchema>;

const jsonSchema = z.toJSONSchema(VoiceNoteSchema);

async function extractVoiceNote(transcript: string): Promise<VoiceNote> {
  const completion = await client.chat.completions.create({
    model: "Qwen/Qwen3.5-9B",
    messages: [
      {
        role: "system",
        content: `Extract structured data from the transcript. Only answer in JSON. Follow this schema: ${JSON.stringify(jsonSchema)}`,
      },
      { role: "user", content: transcript },
    ],
    response_format: {
      type: "json_schema",
      json_schema: {
        name: "voice_note",
        schema: jsonSchema,
      },
    },
  });

  return JSON.parse(completion.choices[0].message.content ?? "{}");
}

const note = await extractVoiceNote(
  "Need to buy groceries today and schedule a meeting with the team for Friday.",
);
console.log(note.title);
console.log(note.actionItems);
```

---

## Complex Nested Schema

```typescript
import Together from "together-ai";
import { z } from "zod";

const client = new Together();

const ArticleSummary = z.object({
  title: z.string(),
  summary: z.string(),
  keyPoints: z.array(z.string()),
  sentiment: z.enum(["positive", "negative", "neutral"]),
  topics: z.array(
    z.object({
      name: z.string(),
      relevance: z.number().describe("Relevance score 0-1"),
    }),
  ),
});

const jsonSchema = z.toJSONSchema(ArticleSummary);

const completion = await client.chat.completions.create({
  model: "Qwen/Qwen3.5-9B",
  messages: [
    {
      role: "system",
      content: `Extract article summary. Only answer in JSON. Schema: ${JSON.stringify(jsonSchema)}`,
    },
    {
      role: "user",
      content:
        "TypeScript 5.8 brings improved type inference and better error messages, making everyday coding more productive.",
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: { name: "article_summary", schema: jsonSchema },
  },
});

const article = JSON.parse(completion.choices[0].message.content ?? "{}");
console.log(`Title: ${article.title}`);
console.log(`Sentiment: ${article.sentiment}`);
```

---

## Regex Mode (Constrained Output)

Constrain output to a regex pattern for classification tasks.

```typescript
import Together from "together-ai";

const client = new Together();

const MAX_TOKENS = 10;

const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  temperature: 0.2,
  max_tokens: MAX_TOKENS,
  messages: [
    {
      role: "system",
      content:
        "Classify the sentiment of the text as positive, neutral, or negative.",
    },
    { role: "user", content: "Wow. I loved the movie!" },
  ],
  response_format: {
    type: "regex",
    // @ts-ignore -- regex type not in SDK types yet
    pattern: "(positive|neutral|negative)",
  },
});

console.log(completion.choices[0].message.content);
// Output: "positive"
```

---

## Vision Model with JSON Output

Extract structured data from images.

```typescript
import Together from "together-ai";
import { z } from "zod";

const client = new Together();

const ImageDescription = z.object({
  description: z.string().describe("What the image shows"),
  objectCount: z.number().describe("Number of main objects"),
  dominantColors: z.array(z.string()).describe("Main colors in the image"),
});

const jsonSchema = z.toJSONSchema(ImageDescription);

const completion = await client.chat.completions.create({
  model: "Qwen/Qwen3-VL-8B-Instruct",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "text",
          text: `Describe this image. Only answer in JSON. Schema: ${JSON.stringify(jsonSchema)}`,
        },
        {
          type: "image_url",
          image_url: { url: "https://example.com/photo.jpg" },
        },
      ],
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: { name: "image_description", schema: jsonSchema },
  },
});

const description = JSON.parse(completion.choices[0].message.content ?? "{}");
console.log(description);
```

---

## Zod v3 vs Zod v4

```typescript
// Zod v4 (recommended): Use built-in z.toJSONSchema()
import { z } from "zod";
const schema = z.object({ name: z.string() });
const jsonSchema = z.toJSONSchema(schema); // Built-in

// Zod v3: Use zodToJsonSchema from separate package
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const schema = z.object({ name: z.string() });
const jsonSchema = zodToJsonSchema(schema); // External package
```

---

_For core concepts, see [SKILL.md](../SKILL.md). For API reference tables, see [reference.md](../reference.md)._

# Together AI SDK -- Tool/Function Calling Examples

> Function calling patterns: tool definitions, tool_choice control, multi-step tool loops, parallel calls. See [SKILL.md](../SKILL.md) for core patterns.

**Related examples:**

- [core.md](core.md) -- Client setup, error handling
- [chat.md](chat.md) -- Chat Completions API
- [streaming.md](streaming.md) -- Streaming responses
- [structured-output.md](structured-output.md) -- Structured outputs with Zod
- [images.md](images.md) -- Image generation, embeddings

---

## Basic Function Calling

```typescript
import Together from "together-ai";

const client = new Together();

const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "What is the weather in Tokyo?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get the current weather for a location",
        parameters: {
          type: "object",
          properties: {
            location: { type: "string", description: "City name" },
            unit: {
              type: "string",
              enum: ["celsius", "fahrenheit"],
              description: "Temperature unit",
            },
          },
          required: ["location"],
          additionalProperties: false,
        },
        strict: true,
      },
    },
  ],
});

const toolCall = completion.choices[0].message.tool_calls?.[0];
if (toolCall) {
  const args = JSON.parse(toolCall.function.arguments);
  console.log(`Call ${toolCall.function.name} with:`, args);
}
```

---

## Multiple Tools

```typescript
import Together from "together-ai";

const client = new Together();

const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [
    {
      role: "system",
      content: "You help users by calling available tools.",
    },
    {
      role: "user",
      content: "What is the weather in London and search for restaurants?",
    },
  ],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get current weather for a city",
        parameters: {
          type: "object",
          properties: {
            location: { type: "string", description: "City name" },
          },
          required: ["location"],
          additionalProperties: false,
        },
      },
    },
    {
      type: "function",
      function: {
        name: "search_restaurants",
        description: "Search for restaurants in a city",
        parameters: {
          type: "object",
          properties: {
            city: { type: "string", description: "City to search" },
            cuisine: { type: "string", description: "Cuisine type" },
          },
          required: ["city"],
          additionalProperties: false,
        },
      },
    },
  ],
});

// Model may return one or more tool calls
const toolCalls = completion.choices[0].message.tool_calls ?? [];
for (const toolCall of toolCalls) {
  const args = JSON.parse(toolCall.function.arguments);
  console.log(`Call ${toolCall.function.name}:`, args);
}
```

---

## Controlling Tool Invocation with `tool_choice`

```typescript
// Force a specific function
const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Tell me about Paris" }],
  tools: [
    /* ... */
  ],
  tool_choice: {
    type: "function",
    function: { name: "get_weather" },
  },
});

// Force at least one tool call (any tool)
const required = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Hello" }],
  tools: [
    /* ... */
  ],
  tool_choice: "required",
});

// Disable tool calling for this request
const noTools = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Hello" }],
  tools: [
    /* ... */
  ],
  tool_choice: "none",
});
```

---

## Multi-Step Tool Loop

Execute tool calls and feed results back to the model.

```typescript
import Together from "together-ai";

const client = new Together();

// Tool implementations
async function getWeather(location: string): Promise<string> {
  return JSON.stringify({ location, temperature: 22, condition: "sunny" });
}

async function searchDatabase(query: string): Promise<string> {
  return JSON.stringify({ results: [{ id: 1, title: `Result: ${query}` }] });
}

const toolImplementations: Record<
  string,
  (args: Record<string, string>) => Promise<string>
> = {
  get_weather: (args) => getWeather(args.location),
  search_database: (args) => searchDatabase(args.query),
};

const tools = [
  {
    type: "function" as const,
    function: {
      name: "get_weather",
      description: "Get weather for a city",
      parameters: {
        type: "object" as const,
        properties: {
          location: { type: "string", description: "City name" },
        },
        required: ["location"],
        additionalProperties: false,
      },
    },
  },
  {
    type: "function" as const,
    function: {
      name: "search_database",
      description: "Search the database",
      parameters: {
        type: "object" as const,
        properties: {
          query: { type: "string", description: "Search query" },
        },
        required: ["query"],
        additionalProperties: false,
      },
    },
  },
];

// Initial request
const messages: Array<{
  role: "system" | "user" | "assistant" | "tool";
  content: string | null;
  tool_calls?: Array<{
    id: string;
    type: "function";
    function: { name: string; arguments: string };
  }>;
  tool_call_id?: string;
  name?: string;
}> = [
  { role: "system", content: "Help users with weather and database queries." },
  { role: "user", content: "What is the weather in London?" },
];

const completion = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages,
  tools,
});

const assistantMessage = completion.choices[0].message;

// If model wants to call tools, execute them and continue
if (assistantMessage.tool_calls && assistantMessage.tool_calls.length > 0) {
  // Add assistant message with tool calls
  messages.push({
    role: "assistant",
    content: assistantMessage.content,
    tool_calls: assistantMessage.tool_calls,
  });

  // Execute each tool and add results
  for (const toolCall of assistantMessage.tool_calls) {
    const args = JSON.parse(toolCall.function.arguments);
    const impl = toolImplementations[toolCall.function.name];
    const result = impl ? await impl(args) : "Unknown tool";

    messages.push({
      role: "tool",
      content: result,
      tool_call_id: toolCall.id,
      name: toolCall.function.name,
    });
  }

  // Get final response without tools
  const finalCompletion = await client.chat.completions.create({
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages,
  });

  console.log("Final answer:", finalCompletion.choices[0].message.content);
}
```

---

## Supported Models for Function Calling

Not all Together AI models support function calling. Currently supported:

- `meta-llama/Llama-3.3-70B-Instruct-Turbo`
- `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`
- `Qwen/Qwen3.5-397B-A17B`
- `Qwen/Qwen3.5-9B`
- `Qwen/Qwen3-Next-80B-A3B-Instruct`
- `Qwen/Qwen2.5-7B-Instruct-Turbo`
- `deepseek-ai/DeepSeek-V3`
- `deepseek-ai/DeepSeek-R1`
- `mistralai/Mistral-Small-24B-Instruct-2501`

Check [Together AI docs](https://docs.together.ai/docs/function-calling) for the latest model support.

---

_For core concepts, see [SKILL.md](../SKILL.md). For API reference tables, see [reference.md](../reference.md)._