# Azure AI Voice Live SDK - API Reference

## Table of Contents
- [connect() Function](#connect-function)
- [VoiceLiveConnection](#voiceliveconnection)
- [SessionResource](#sessionresource)
- [ResponseResource](#responseresource)
- [InputAudioBufferResource](#inputaudiobufferresource)
- [OutputAudioBufferResource](#outputaudiobufferresource)
- [ConversationResource](#conversationresource)
- [TranscriptionSessionResource](#transcriptionsessionresource)
- [WebsocketConnectionOptions](#websocketconnectionoptions)
- [Exceptions](#exceptions)

---

## connect() Function

Creates an async context manager for WebSocket connections.

```python
from azure.ai.voicelive.aio import connect

async with connect(
    credential: Union[AzureKeyCredential, AsyncTokenCredential],
    endpoint: str,
    api_version: str = "2025-10-01",
    model: Optional[str] = None,
    query: Optional[Mapping[str, Any]] = None,
    headers: Optional[Mapping[str, Any]] = None,
    connection_options: Optional[WebsocketConnectionOptions] = None,
    credential_scopes: Optional[List[str]] = None,
    **kwargs
) -> VoiceLiveConnection:
    ...
```

### Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `credential` | `AzureKeyCredential` or `AsyncTokenCredential` | Yes | Authentication credential |
| `endpoint` | `str` | Yes | Service endpoint URL |
| `api_version` | `str` | No | API version (default: "2025-10-01") |
| `model` | `str` | Sometimes | Model identifier (required unless using Agent scenario) |
| `query` | `Mapping[str, Any]` | No | Additional URL query parameters |
| `headers` | `Mapping[str, Any]` | No | Additional HTTP headers |
| `connection_options` | `WebsocketConnectionOptions` | No | WebSocket transport options |
| `credential_scopes` | `List[str]` | No | OAuth scopes (default: `["https://ai.azure.com/.default"]`) |

---

## VoiceLiveConnection

Main connection class with resource accessors.

### Properties

| Property | Type | Description |
|----------|------|-------------|
| `session` | `SessionResource` | Session configuration |
| `response` | `ResponseResource` | Response management |
| `input_audio_buffer` | `InputAudioBufferResource` | Audio input buffer |
| `output_audio_buffer` | `OutputAudioBufferResource` | Audio output buffer |
| `conversation` | `ConversationResource` | Conversation items |
| `transcription_session` | `TranscriptionSessionResource` | Transcription config |

### Methods

```python
async def recv() -> ServerEvent:
    """Receive and parse the next typed server event."""

async def recv_bytes() -> bytes:
    """Receive raw bytes from the connection."""

async def send(event: Union[Mapping[str, Any], ClientEvent]) -> None:
    """Send an event to the server."""

async def close(*, code: int = 1000, reason: str = "") -> None:
    """Close the WebSocket connection."""

async def __aiter__() -> AsyncIterator[ServerEvent]:
    """Iterate over server events until connection closes."""
```

---

## SessionResource

Manage session configuration.

### Methods

```python
async def update(
    *,
    session: Union[Mapping[str, Any], RequestSession],
    event_id: Optional[str] = None
) -> None:
    """Update session configuration."""
```

### RequestSession Fields

| Field | Type | Description |
|-------|------|-------------|
| `instructions` | `str` | System prompt for the model |
| `modalities` | `List[Modality]` | `["text"]`, `["audio"]`, or `["text", "audio"]` |
| `voice` | `Voice` | Voice for audio output |
| `input_audio_format` | `InputAudioFormat` | Audio input format |
| `output_audio_format` | `OutputAudioFormat` | Audio output format |
| `turn_detection` | `TurnDetection` | VAD configuration or `None` for manual |
| `tools` | `List[Tool]` | Function tools |
| `tool_choice` | `ToolChoice` | `"auto"`, `"none"`, `"required"`, or specific function |
| `temperature` | `float` | Model temperature (0.6-1.2) |
| `max_response_output_tokens` | `int` or `"inf"` | Max tokens per response |
| `input_audio_transcription` | `AudioInputTranscriptionOptions` | Transcription settings |

---

## ResponseResource

Manage model responses.

### Methods

```python
async def create(
    *,
    response: Optional[Union[ResponseCreateParams, Mapping[str, Any]]] = None,
    event_id: Optional[str] = None,
    additional_instructions: Optional[str] = None
) -> None:
    """Create a response (trigger inference)."""

async def cancel(
    *,
    response_id: Optional[str] = None,
    event_id: Optional[str] = None
) -> None:
    """Cancel an in-progress response."""
```

### ResponseCreateParams Fields

| Field | Type | Description |
|-------|------|-------------|
| `modalities` | `List[Modality]` | Override session modalities |
| `instructions` | `str` | Override session instructions |
| `voice` | `Voice` | Override session voice |
| `temperature` | `float` | Override temperature |
| `max_response_output_tokens` | `int` | Override max tokens |
| `conversation` | `str` | `"auto"` or `"none"` |

---

## InputAudioBufferResource

Manage audio input buffer.

### Methods

```python
async def append(
    *,
    audio: str,  # Base64-encoded audio
    event_id: Optional[str] = None
) -> None:
    """Append audio data to the input buffer."""

async def commit(
    *,
    event_id: Optional[str] = None
) -> None:
    """Commit the buffer as a user message."""

async def clear(
    *,
    event_id: Optional[str] = None
) -> None:
    """Clear the input buffer without committing."""
```

---

## OutputAudioBufferResource

Manage audio output buffer.

### Methods

```python
async def clear(
    *,
    event_id: Optional[str] = None
) -> None:
    """Clear pending audio output (for interrupts)."""
```

---

## ConversationResource

Manage conversation state.

### Properties

| Property | Type | Description |
|----------|------|-------------|
| `item` | `ConversationItemResource` | Item operations |

---

## ConversationItemResource

CRUD operations on conversation items.

### Methods

```python
async def create(
    *,
    item: Union[ConversationRequestItem, Mapping[str, Any]],
    previous_item_id: Optional[str] = None,
    event_id: Optional[str] = None
) -> None:
    """Create a new conversation item."""

async def delete(
    *,
    item_id: str,
    event_id: Optional[str] = None
) -> None:
    """Delete a conversation item."""

async def retrieve(
    *,
    item_id: str,
    event_id: Optional[str] = None
) -> None:
    """Retrieve item details (server responds with event)."""

async def truncate(
    *,
    item_id: str,
    audio_end_ms: int,
    content_index: int,
    event_id: Optional[str] = None
) -> None:
    """Truncate audio at specified time."""
```

### ConversationRequestItem Types

```python
# User/Assistant/System message
{
    "type": "message",
    "role": "user" | "assistant" | "system",
    "content": [
        {"type": "input_text", "text": "..."},
        {"type": "input_audio", "audio": "base64..."},
        {"type": "text", "text": "..."},
        {"type": "audio", "audio": "base64...", "transcript": "..."}
    ]
}

# Function call (from model)
{
    "type": "function_call",
    "call_id": "...",
    "name": "function_name",
    "arguments": "{...}"
}

# Function output (from client)
{
    "type": "function_call_output",
    "call_id": "...",
    "output": "{...}"
}
```

---

## TranscriptionSessionResource

Configure input transcription.

### Methods

```python
async def update(
    *,
    session: Mapping[str, Any],
    event_id: Optional[str] = None
) -> None:
    """Update transcription session configuration."""
```

---

## WebsocketConnectionOptions

Transport configuration for the WebSocket connection.

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `compression` | `bool` or `int` | None | Enable per-message compression |
| `max_msg_size` | `int` | 4MB | Maximum message size |
| `heartbeat` | `float` | 30 | Keep-alive ping interval (seconds) |
| `autoclose` | `bool` | True | Auto-close on close frame |
| `autoping` | `bool` | True | Auto-respond to pings |
| `receive_timeout` | `float` | None | Message receive timeout |
| `close_timeout` | `float` | None | Close handshake timeout |
| `handshake_timeout` | `float` | None | Connection establishment timeout |
| `vendor_options` | `Mapping` | None | Implementation-specific options |

---

## Exceptions

```python
from azure.ai.voicelive.aio import ConnectionError, ConnectionClosed

class ConnectionError(AzureError):
    """Base WebSocket connection error."""

class ConnectionClosed(ConnectionError):
    """WebSocket connection was closed."""
    code: int      # Close code
    reason: str    # Close reason
```

### Common Close Codes

| Code | Meaning |
|------|---------|
| 1000 | Normal closure |
| 1001 | Going away |
| 1006 | Abnormal closure |
| 1008 | Policy violation |
| 1011 | Server error |

# Azure AI Voice Live SDK - Examples

## Table of Contents
- [Basic Voice Assistant](#basic-voice-assistant)
- [Function Calling](#function-calling)
- [Manual Turn Control](#manual-turn-control)
- [Audio File Processing](#audio-file-processing)
- [Interrupt Handling](#interrupt-handling)
- [Multi-modal (Text + Audio)](#multi-modal-text--audio)
- [Azure Voice Integration](#azure-voice-integration)
- [Avatar Integration](#avatar-integration)
- [Transcription Only](#transcription-only)

---

## Basic Voice Assistant

Complete voice assistant with Server VAD.

```python
import asyncio
import base64
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def voice_assistant():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        # Configure session
        await conn.session.update(session={
            "instructions": "You are a helpful voice assistant. Be concise.",
            "modalities": ["text", "audio"],
            "voice": "alloy",
            "turn_detection": {
                "type": "server_vad",
                "threshold": 0.5,
                "silence_duration_ms": 500
            },
            "input_audio_transcription": {
                "model": "whisper-1"
            }
        })
        
        # Start microphone input (pseudo-code)
        mic_task = asyncio.create_task(stream_microphone(conn))
        
        # Process events
        async for event in conn:
            match event.type:
                case "session.created":
                    print("Session ready")
                
                case "input_audio_buffer.speech_started":
                    print("🎤 Listening...")
                
                case "conversation.item.input_audio_transcription.completed":
                    print(f"You: {event.transcript}")
                
                case "response.audio.delta":
                    audio = base64.b64decode(event.delta)
                    await play_audio(audio)
                
                case "response.audio_transcript.done":
                    print(f"Assistant: {event.transcript}")
                
                case "error":
                    print(f"Error: {event.error.message}")
                    break

async def stream_microphone(conn):
    """Stream microphone audio to the connection."""
    async for chunk in read_microphone():  # Your audio capture
        b64 = base64.b64encode(chunk).decode()
        await conn.input_audio_buffer.append(audio=b64)

asyncio.run(voice_assistant())
```

---

## Function Calling

Voice assistant with tool use.

```python
import asyncio
import json
import base64
from azure.ai.voicelive.aio import connect
from azure.ai.voicelive.models import FunctionTool
from azure.core.credentials import AzureKeyCredential

# Define tools
TOOLS = [
    FunctionTool(
        type="function",
        name="get_weather",
        description="Get current weather for a location",
        parameters={
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and state, e.g. 'San Francisco, CA'"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "default": "fahrenheit"
                }
            },
            "required": ["location"]
        }
    ),
    FunctionTool(
        type="function",
        name="set_reminder",
        description="Set a reminder for the user",
        parameters={
            "type": "object",
            "properties": {
                "message": {"type": "string"},
                "time": {"type": "string", "description": "ISO 8601 datetime"}
            },
            "required": ["message", "time"]
        }
    )
]

def handle_function_call(name: str, arguments: str) -> dict:
    """Execute function and return result."""
    args = json.loads(arguments)
    
    if name == "get_weather":
        # Mock weather API
        return {
            "location": args["location"],
            "temperature": 72,
            "unit": args.get("unit", "fahrenheit"),
            "conditions": "sunny"
        }
    elif name == "set_reminder":
        # Mock reminder service
        return {"status": "success", "reminder_id": "123"}
    else:
        return {"error": f"Unknown function: {name}"}

async def function_calling_assistant():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        await conn.session.update(session={
            "instructions": "You can check weather and set reminders.",
            "modalities": ["text", "audio"],
            "voice": "alloy",
            "tools": TOOLS,
            "tool_choice": "auto"
        })
        
        async for event in conn:
            match event.type:
                case "response.function_call_arguments.done":
                    # Execute the function
                    result = handle_function_call(event.name, event.arguments)
                    
                    # Send result back
                    await conn.conversation.item.create(item={
                        "type": "function_call_output",
                        "call_id": event.call_id,
                        "output": json.dumps(result)
                    })
                    
                    # Continue the conversation
                    await conn.response.create()
                
                case "response.audio.delta":
                    audio = base64.b64decode(event.delta)
                    await play_audio(audio)
                
                case "response.done":
                    if event.response.status == "completed":
                        print("Response complete")

asyncio.run(function_calling_assistant())
```

---

## Manual Turn Control

Push-to-talk style without VAD.

```python
import asyncio
import base64
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def push_to_talk():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        # Disable VAD for manual control
        await conn.session.update(session={
            "instructions": "You are a helpful assistant.",
            "modalities": ["text", "audio"],
            "voice": "alloy",
            "turn_detection": None  # Disable VAD
        })
        
        # Simulate push-to-talk
        while True:
            input("Press Enter to start recording...")
            
            # Record audio (simulate with chunks)
            chunks = await record_audio_until_release()
            
            # Send all audio
            for chunk in chunks:
                b64 = base64.b64encode(chunk).decode()
                await conn.input_audio_buffer.append(audio=b64)
            
            # Commit and request response
            await conn.input_audio_buffer.commit()
            await conn.response.create()
            
            # Wait for response
            async for event in conn:
                if event.type == "response.audio.delta":
                    audio = base64.b64decode(event.delta)
                    await play_audio(audio)
                elif event.type == "response.done":
                    break

asyncio.run(push_to_talk())
```

---

## Audio File Processing

Process an audio file and get a response.

```python
import asyncio
import base64
from pathlib import Path
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def process_audio_file(audio_path: str):
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        await conn.session.update(session={
            "instructions": "Respond to the audio message.",
            "modalities": ["text", "audio"],
            "voice": "alloy",
            "turn_detection": None,
            "input_audio_transcription": {"model": "whisper-1"}
        })
        
        # Read and send audio file
        audio_data = Path(audio_path).read_bytes()
        
        # Send in chunks (24kHz * 2 bytes * 0.1s = 4800 bytes)
        chunk_size = 4800
        for i in range(0, len(audio_data), chunk_size):
            chunk = audio_data[i:i + chunk_size]
            b64 = base64.b64encode(chunk).decode()
            await conn.input_audio_buffer.append(audio=b64)
        
        # Commit and request response
        await conn.input_audio_buffer.commit()
        await conn.response.create()
        
        # Collect response
        response_audio = bytearray()
        response_text = ""
        user_transcript = ""
        
        async for event in conn:
            match event.type:
                case "conversation.item.input_audio_transcription.completed":
                    user_transcript = event.transcript
                case "response.audio.delta":
                    response_audio.extend(base64.b64decode(event.delta))
                case "response.audio_transcript.done":
                    response_text = event.transcript
                case "response.done":
                    break
        
        return {
            "user_said": user_transcript,
            "assistant_said": response_text,
            "audio": bytes(response_audio)
        }

result = asyncio.run(process_audio_file("input.pcm"))
print(f"User: {result['user_said']}")
print(f"Assistant: {result['assistant_said']}")
Path("output.pcm").write_bytes(result['audio'])
```

---

## Interrupt Handling

Handle user interruptions gracefully.

```python
import asyncio
import base64
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def interruptible_assistant():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        await conn.session.update(session={
            "instructions": "You are a helpful assistant.",
            "modalities": ["text", "audio"],
            "voice": "alloy",
            "turn_detection": {
                "type": "server_vad",
                "threshold": 0.5,
                "silence_duration_ms": 500
            }
        })
        
        is_responding = False
        
        async for event in conn:
            match event.type:
                case "response.created":
                    is_responding = True
                
                case "response.done":
                    is_responding = False
                
                case "input_audio_buffer.speech_started":
                    if is_responding:
                        # User interrupted - stop current response
                        print("🛑 Interrupt detected!")
                        await conn.response.cancel()
                        await conn.output_audio_buffer.clear()
                
                case "response.audio.delta":
                    if is_responding:
                        audio = base64.b64decode(event.delta)
                        await play_audio(audio)

asyncio.run(interruptible_assistant())
```

---

## Multi-modal (Text + Audio)

Send text context, receive audio response.

```python
import asyncio
import base64
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def multimodal_assistant():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        await conn.session.update(session={
            "instructions": "You are a helpful assistant.",
            "modalities": ["text", "audio"],
            "voice": "alloy",
            "turn_detection": None
        })
        
        # Add context via text
        await conn.conversation.item.create(item={
            "type": "message",
            "role": "system",
            "content": [{"type": "input_text", "text": "The user's name is Alice."}]
        })
        
        # Add user message as text
        await conn.conversation.item.create(item={
            "type": "message",
            "role": "user",
            "content": [{"type": "input_text", "text": "What's my name?"}]
        })
        
        # Request audio response
        await conn.response.create()
        
        async for event in conn:
            if event.type == "response.audio.delta":
                audio = base64.b64decode(event.delta)
                await play_audio(audio)
            elif event.type == "response.done":
                break

asyncio.run(multimodal_assistant())
```

---

## Azure Voice Integration

Use Azure Text-to-Speech voices.

```python
import asyncio
from azure.ai.voicelive.aio import connect
from azure.ai.voicelive.models import AzureStandardVoice, AzureCustomVoice
from azure.core.credentials import AzureKeyCredential

async def azure_voice_assistant():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        # Use Azure standard voice
        await conn.session.update(session={
            "instructions": "You are a helpful assistant.",
            "modalities": ["text", "audio"],
            "voice": AzureStandardVoice(
                type="azure-standard",
                name="en-US-JennyNeural"
            )
        })
        
        # Or use custom voice
        # await conn.session.update(session={
        #     "voice": AzureCustomVoice(
        #         type="azure-custom",
        #         endpoint_id="your-custom-voice-endpoint",
        #         name="YourCustomVoice"
        #     )
        # })
        
        async for event in conn:
            # ... handle events
            pass

asyncio.run(azure_voice_assistant())
```

---

## Avatar Integration

Connect to Azure Avatar for visual output.

```python
import asyncio
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def avatar_assistant():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        await conn.session.update(session={
            "instructions": "You are a helpful assistant.",
            "modalities": ["text", "audio", "avatar"],
            "voice": "alloy",
            "avatar": {
                "type": "video-avatar",
                "character": "lisa",
                "output_protocol": "webrtc"
            }
        })
        
        # Connect avatar
        await conn.send({
            "type": "session.avatar.connect"
        })
        
        async for event in conn:
            match event.type:
                case "session.avatar.connecting":
                    ice_servers = event.ice_servers
                    # Use ice_servers for WebRTC connection
                    print(f"Avatar connecting with {len(ice_servers)} ICE servers")
                
                case "response.audio.delta":
                    # Audio is streamed via WebRTC, not this event
                    pass

asyncio.run(avatar_assistant())
```

---

## Transcription Only

Speech-to-text without AI response.

```python
import asyncio
import base64
from azure.ai.voicelive.aio import connect
from azure.core.credentials import AzureKeyCredential

async def transcription_only():
    async with connect(
        endpoint="https://eastus.api.cognitive.microsoft.com",
        credential=AzureKeyCredential("YOUR_KEY"),
        model="gpt-4o-realtime-preview"
    ) as conn:
        await conn.session.update(session={
            "modalities": ["text"],  # No audio output
            "turn_detection": {
                "type": "server_vad",
                "threshold": 0.5,
                "silence_duration_ms": 1000
            },
            "input_audio_transcription": {
                "model": "whisper-1"
            }
        })
        
        # Stream microphone
        mic_task = asyncio.create_task(stream_microphone(conn))
        
        transcripts = []
        
        async for event in conn:
            match event.type:
                case "conversation.item.input_audio_transcription.delta":
                    print(event.delta, end="", flush=True)
                
                case "conversation.item.input_audio_transcription.completed":
                    print()  # Newline
                    transcripts.append(event.transcript)
        
        return transcripts

asyncio.run(transcription_only())
```

# Azure AI Voice Live SDK - Models Reference

## Table of Contents
- [Enums](#enums)
- [Client Events](#client-events)
- [Server Events](#server-events)
- [Session Models](#session-models)
- [Conversation Items](#conversation-items)
- [Content Parts](#content-parts)
- [Tools](#tools)
- [Voice Models](#voice-models)
- [Turn Detection](#turn-detection)
- [Response Models](#response-models)
- [Avatar Models](#avatar-models)

---

## Enums

### Modality
```python
class Modality(str, Enum):
    TEXT = "text"
    AUDIO = "audio"
    ANIMATION = "animation"
    AVATAR = "avatar"
```

### OpenAIVoiceName
```python
class OpenAIVoiceName(str, Enum):
    ALLOY = "alloy"
    ASH = "ash"
    BALLAD = "ballad"
    CORAL = "coral"
    ECHO = "echo"
    SAGE = "sage"
    SHIMMER = "shimmer"
    VERSE = "verse"
    MARIN = "marin"
    CEDAR = "cedar"
```

### InputAudioFormat
```python
class InputAudioFormat(str, Enum):
    PCM16 = "pcm16"           # 24kHz default
    G711_ULAW = "g711_ulaw"   # 8kHz
    G711_ALAW = "g711_alaw"   # 8kHz
```

### OutputAudioFormat
```python
class OutputAudioFormat(str, Enum):
    PCM16 = "pcm16"               # 24kHz
    PCM16_8000_HZ = "pcm16-8000hz"
    PCM16_16000_HZ = "pcm16-16000hz"
    G711_ULAW = "g711_ulaw"       # 8kHz
    G711_ALAW = "g711_alaw"       # 8kHz
```

### TurnDetectionType
```python
class TurnDetectionType(str, Enum):
    SERVER_VAD = "server_vad"
    AZURE_SEMANTIC_VAD = "azure_semantic_vad"
    AZURE_SEMANTIC_VAD_EN = "azure_semantic_vad_en"
    AZURE_SEMANTIC_VAD_MULTILINGUAL = "azure_semantic_vad_multilingual"
```

### MessageRole
```python
class MessageRole(str, Enum):
    SYSTEM = "system"
    USER = "user"
    ASSISTANT = "assistant"
```

### ItemType
```python
class ItemType(str, Enum):
    MESSAGE = "message"
    FUNCTION_CALL = "function_call"
    FUNCTION_CALL_OUTPUT = "function_call_output"
    MCP_LIST_TOOLS = "mcp_list_tools"
    MCP_CALL = "mcp_call"
    MCP_APPROVAL_REQUEST = "mcp_approval_request"
    MCP_APPROVAL_RESPONSE = "mcp_approval_response"
```

### ContentPartType
```python
class ContentPartType(str, Enum):
    INPUT_TEXT = "input_text"
    INPUT_AUDIO = "input_audio"
    INPUT_IMAGE = "input_image"
    TEXT = "text"
    AUDIO = "audio"
```

### ToolType
```python
class ToolType(str, Enum):
    FUNCTION = "function"
    MCP = "mcp"
```

### ToolChoiceLiteral
```python
class ToolChoiceLiteral(str, Enum):
    AUTO = "auto"
    NONE = "none"
    REQUIRED = "required"
```

### ResponseStatus
```python
class ResponseStatus(str, Enum):
    COMPLETED = "completed"
    CANCELLED = "cancelled"
    FAILED = "failed"
    INCOMPLETE = "incomplete"
    IN_PROGRESS = "in_progress"
```

### ClientEventType
```python
class ClientEventType(str, Enum):
    SESSION_UPDATE = "session.update"
    INPUT_AUDIO_BUFFER_APPEND = "input_audio_buffer.append"
    INPUT_AUDIO_BUFFER_COMMIT = "input_audio_buffer.commit"
    INPUT_AUDIO_BUFFER_CLEAR = "input_audio_buffer.clear"
    INPUT_AUDIO_TURN_START = "input_audio.turn.start"
    INPUT_AUDIO_TURN_APPEND = "input_audio.turn.append"
    INPUT_AUDIO_TURN_END = "input_audio.turn.end"
    INPUT_AUDIO_TURN_CANCEL = "input_audio.turn.cancel"
    INPUT_AUDIO_CLEAR = "input_audio.clear"
    CONVERSATION_ITEM_CREATE = "conversation.item.create"
    CONVERSATION_ITEM_RETRIEVE = "conversation.item.retrieve"
    CONVERSATION_ITEM_TRUNCATE = "conversation.item.truncate"
    CONVERSATION_ITEM_DELETE = "conversation.item.delete"
    RESPONSE_CREATE = "response.create"
    RESPONSE_CANCEL = "response.cancel"
    SESSION_AVATAR_CONNECT = "session.avatar.connect"
    MCP_APPROVAL_RESPONSE = "mcp_approval_response"
```

### ServerEventType
```python
class ServerEventType(str, Enum):
    ERROR = "error"
    SESSION_AVATAR_CONNECTING = "session.avatar.connecting"
    SESSION_CREATED = "session.created"
    SESSION_UPDATED = "session.updated"
    CONVERSATION_ITEM_INPUT_AUDIO_TRANSCRIPTION_COMPLETED = "conversation.item.input_audio_transcription.completed"
    CONVERSATION_ITEM_INPUT_AUDIO_TRANSCRIPTION_DELTA = "conversation.item.input_audio_transcription.delta"
    CONVERSATION_ITEM_INPUT_AUDIO_TRANSCRIPTION_FAILED = "conversation.item.input_audio_transcription.failed"
    CONVERSATION_ITEM_CREATED = "conversation.item.created"
    CONVERSATION_ITEM_RETRIEVED = "conversation.item.retrieved"
    CONVERSATION_ITEM_TRUNCATED = "conversation.item.truncated"
    CONVERSATION_ITEM_DELETED = "conversation.item.deleted"
    INPUT_AUDIO_BUFFER_COMMITTED = "input_audio_buffer.committed"
    INPUT_AUDIO_BUFFER_CLEARED = "input_audio_buffer.cleared"
    INPUT_AUDIO_BUFFER_SPEECH_STARTED = "input_audio_buffer.speech_started"
    INPUT_AUDIO_BUFFER_SPEECH_STOPPED = "input_audio_buffer.speech_stopped"
    RESPONSE_CREATED = "response.created"
    RESPONSE_DONE = "response.done"
    RESPONSE_OUTPUT_ITEM_ADDED = "response.output_item.added"
    RESPONSE_OUTPUT_ITEM_DONE = "response.output_item.done"
    RESPONSE_CONTENT_PART_ADDED = "response.content_part.added"
    RESPONSE_CONTENT_PART_DONE = "response.content_part.done"
    RESPONSE_TEXT_DELTA = "response.text.delta"
    RESPONSE_TEXT_DONE = "response.text.done"
    RESPONSE_AUDIO_TRANSCRIPT_DELTA = "response.audio_transcript.delta"
    RESPONSE_AUDIO_TRANSCRIPT_DONE = "response.audio_transcript.done"
    RESPONSE_AUDIO_DELTA = "response.audio.delta"
    RESPONSE_AUDIO_DONE = "response.audio.done"
    RESPONSE_FUNCTION_CALL_ARGUMENTS_DELTA = "response.function_call_arguments.delta"
    RESPONSE_FUNCTION_CALL_ARGUMENTS_DONE = "response.function_call_arguments.done"
    # MCP events
    MCP_LIST_TOOLS_IN_PROGRESS = "mcp_list_tools.in_progress"
    MCP_LIST_TOOLS_COMPLETED = "mcp_list_tools.completed"
    MCP_LIST_TOOLS_FAILED = "mcp_list_tools.failed"
    RESPONSE_MCP_CALL_ARGUMENTS_DELTA = "response.mcp_call_arguments.delta"
    RESPONSE_MCP_CALL_ARGUMENTS_DONE = "response.mcp_call_arguments.done"
    RESPONSE_MCP_CALL_IN_PROGRESS = "response.mcp_call.in_progress"
    RESPONSE_MCP_CALL_COMPLETED = "response.mcp_call.completed"
    RESPONSE_MCP_CALL_FAILED = "response.mcp_call.failed"
    # Animation events
    RESPONSE_ANIMATION_BLENDSHAPES_DELTA = "response.animation_blendshapes.delta"
    RESPONSE_ANIMATION_BLENDSHAPES_DONE = "response.animation_blendshapes.done"
    RESPONSE_ANIMATION_VISEME_DELTA = "response.animation_viseme.delta"
    RESPONSE_ANIMATION_VISEME_DONE = "response.animation_viseme.done"
    RESPONSE_AUDIO_TIMESTAMP_DELTA = "response.audio_timestamp.delta"
    RESPONSE_AUDIO_TIMESTAMP_DONE = "response.audio_timestamp.done"
```

---

## Client Events

### ClientEventSessionUpdate
```python
class ClientEventSessionUpdate(Model):
    type: Literal["session.update"]
    event_id: Optional[str]
    session: RequestSession
```

### ClientEventInputAudioBufferAppend
```python
class ClientEventInputAudioBufferAppend(Model):
    type: Literal["input_audio_buffer.append"]
    event_id: Optional[str]
    audio: str  # Base64-encoded audio
```

### ClientEventInputAudioBufferCommit
```python
class ClientEventInputAudioBufferCommit(Model):
    type: Literal["input_audio_buffer.commit"]
    event_id: Optional[str]
```

### ClientEventInputAudioBufferClear
```python
class ClientEventInputAudioBufferClear(Model):
    type: Literal["input_audio_buffer.clear"]
    event_id: Optional[str]
```

### ClientEventConversationItemCreate
```python
class ClientEventConversationItemCreate(Model):
    type: Literal["conversation.item.create"]
    event_id: Optional[str]
    previous_item_id: Optional[str]
    item: ConversationRequestItem
```

### ClientEventConversationItemDelete
```python
class ClientEventConversationItemDelete(Model):
    type: Literal["conversation.item.delete"]
    event_id: Optional[str]
    item_id: str
```

### ClientEventConversationItemTruncate
```python
class ClientEventConversationItemTruncate(Model):
    type: Literal["conversation.item.truncate"]
    event_id: Optional[str]
    item_id: str
    content_index: int
    audio_end_ms: int
```

### ClientEventResponseCreate
```python
class ClientEventResponseCreate(Model):
    type: Literal["response.create"]
    event_id: Optional[str]
    response: Optional[ResponseCreateParams]
    additional_instructions: Optional[str]
```

### ClientEventResponseCancel
```python
class ClientEventResponseCancel(Model):
    type: Literal["response.cancel"]
    event_id: Optional[str]
    response_id: Optional[str]
```

---

## Server Events

### ServerEventSessionCreated
```python
class ServerEventSessionCreated(Model):
    type: Literal["session.created"]
    event_id: str
    session: ResponseSession
```

### ServerEventSessionUpdated
```python
class ServerEventSessionUpdated(Model):
    type: Literal["session.updated"]
    event_id: str
    session: ResponseSession
```

### ServerEventError
```python
class ServerEventError(Model):
    type: Literal["error"]
    event_id: str
    error: ServerEventErrorDetails

class ServerEventErrorDetails(Model):
    type: str
    code: Optional[str]
    message: str
    param: Optional[str]
```

### ServerEventInputAudioBufferSpeechStarted
```python
class ServerEventInputAudioBufferSpeechStarted(Model):
    type: Literal["input_audio_buffer.speech_started"]
    event_id: str
    audio_start_ms: int
    item_id: str
```

### ServerEventInputAudioBufferSpeechStopped
```python
class ServerEventInputAudioBufferSpeechStopped(Model):
    type: Literal["input_audio_buffer.speech_stopped"]
    event_id: str
    audio_end_ms: int
    item_id: str
```

### ServerEventConversationItemInputAudioTranscriptionCompleted
```python
class ServerEventConversationItemInputAudioTranscriptionCompleted(Model):
    type: Literal["conversation.item.input_audio_transcription.completed"]
    event_id: str
    item_id: str
    content_index: int
    transcript: str
```

### ServerEventConversationItemInputAudioTranscriptionDelta
```python
class ServerEventConversationItemInputAudioTranscriptionDelta(Model):
    type: Literal["conversation.item.input_audio_transcription.delta"]
    event_id: str
    item_id: str
    content_index: int
    delta: str
```

### ServerEventResponseCreated
```python
class ServerEventResponseCreated(Model):
    type: Literal["response.created"]
    event_id: str
    response: Response
```

### ServerEventResponseDone
```python
class ServerEventResponseDone(Model):
    type: Literal["response.done"]
    event_id: str
    response: Response
```

### ServerEventResponseAudioDelta
```python
class ServerEventResponseAudioDelta(Model):
    type: Literal["response.audio.delta"]
    event_id: str
    response_id: str
    item_id: str
    output_index: int
    content_index: int
    delta: str  # Base64-encoded audio
```

### ServerEventResponseAudioTranscriptDelta
```python
class ServerEventResponseAudioTranscriptDelta(Model):
    type: Literal["response.audio_transcript.delta"]
    event_id: str
    response_id: str
    item_id: str
    output_index: int
    content_index: int
    delta: str
```

### ServerEventResponseAudioTranscriptDone
```python
class ServerEventResponseAudioTranscriptDone(Model):
    type: Literal["response.audio_transcript.done"]
    event_id: str
    response_id: str
    item_id: str
    output_index: int
    content_index: int
    transcript: str
```

### ServerEventResponseFunctionCallArgumentsDelta
```python
class ServerEventResponseFunctionCallArgumentsDelta(Model):
    type: Literal["response.function_call_arguments.delta"]
    event_id: str
    response_id: str
    item_id: str
    output_index: int
    call_id: str
    delta: str
```

### ServerEventResponseFunctionCallArgumentsDone
```python
class ServerEventResponseFunctionCallArgumentsDone(Model):
    type: Literal["response.function_call_arguments.done"]
    event_id: str
    response_id: str
    item_id: str
    output_index: int
    call_id: str
    name: str
    arguments: str
```

---

## Session Models

### RequestSession
```python
class RequestSession(Model):
    instructions: Optional[str]
    modalities: Optional[List[Modality]]
    voice: Optional[Voice]  # str, OpenAIVoiceName, OpenAIVoice, or AzureVoice
    input_audio_format: Optional[InputAudioFormat]
    output_audio_format: Optional[OutputAudioFormat]
    turn_detection: Optional[TurnDetection]
    tools: Optional[List[Tool]]
    tool_choice: Optional[ToolChoice]
    temperature: Optional[float]
    max_response_output_tokens: Optional[Union[int, Literal["inf"]]]
    input_audio_transcription: Optional[AudioInputTranscriptionOptions]
```

### ResponseSession
```python
class ResponseSession(Model):
    id: str
    object: str
    model: str
    expires_at: int
    modalities: List[Modality]
    instructions: Optional[str]
    voice: Optional[Voice]
    input_audio_format: InputAudioFormat
    output_audio_format: OutputAudioFormat
    turn_detection: Optional[TurnDetection]
    tools: List[Tool]
    tool_choice: ToolChoice
    temperature: float
    max_response_output_tokens: Optional[int]
```

### AudioInputTranscriptionOptions
```python
class AudioInputTranscriptionOptions(Model):
    model: str  # e.g., "whisper-1"
```

---

## Conversation Items

### ConversationRequestItem (Union Type)
```python
# Can be one of:
- SystemMessageItem
- UserMessageItem
- AssistantMessageItem
- FunctionCallItem
- FunctionCallOutputItem
```

### MessageItem Base
```python
class MessageItem(Model):
    type: Literal["message"]
    id: Optional[str]
    role: MessageRole
    content: List[ContentPart]
    status: Optional[ItemParamStatus]
```

### FunctionCallItem
```python
class FunctionCallItem(Model):
    type: Literal["function_call"]
    id: Optional[str]
    call_id: str
    name: str
    arguments: str
    status: Optional[ItemParamStatus]
```

### FunctionCallOutputItem
```python
class FunctionCallOutputItem(Model):
    type: Literal["function_call_output"]
    id: Optional[str]
    call_id: str
    output: str
```

---

## Content Parts

### InputTextContentPart
```python
class InputTextContentPart(Model):
    type: Literal["input_text"]
    text: str
```

### InputAudioContentPart
```python
class InputAudioContentPart(Model):
    type: Literal["input_audio"]
    audio: str  # Base64
    transcript: Optional[str]
```

### RequestTextContentPart
```python
class RequestTextContentPart(Model):
    type: Literal["text"]
    text: str
```

### RequestAudioContentPart
```python
class RequestAudioContentPart(Model):
    type: Literal["audio"]
    audio: str  # Base64
    transcript: Optional[str]
```

### RequestImageContentPart
```python
class RequestImageContentPart(Model):
    type: Literal["input_image"]
    url: Optional[str]
    base64: Optional[str]
    detail: Optional[RequestImageContentPartDetail]  # "auto", "low", "high"
```

---

## Tools

### FunctionTool
```python
class FunctionTool(Model):
    type: Literal["function"]
    name: str
    description: Optional[str]
    parameters: Optional[dict]  # JSON Schema
```

### MCPTool
```python
class MCPTool(Model):
    type: Literal["mcp"]
    server_label: str
    require_approval: Optional[MCPApprovalType]  # "never" or "always"
```

### MCPServer
```python
class MCPServer(Model):
    type: Literal["url"]
    url: str
    name: str
    tool_configuration: Optional[dict]
```

### ToolChoiceSelection
```python
class ToolChoiceSelection(Model):
    type: Literal["function"]
    name: str
```

---

## Voice Models

### OpenAIVoice
```python
class OpenAIVoice(Model):
    type: Literal["openai"]
    name: OpenAIVoiceName
```

### AzureStandardVoice
```python
class AzureStandardVoice(Model):
    type: Literal["azure-standard"]
    name: str  # e.g., "en-US-JennyNeural"
```

### AzureCustomVoice
```python
class AzureCustomVoice(Model):
    type: Literal["azure-custom"]
    endpoint_id: str
    name: str
```

### AzurePersonalVoice
```python
class AzurePersonalVoice(Model):
    type: Literal["azure-personal"]
    speaker_profile_id: str
    model: Optional[PersonalVoiceModels]
```

---

## Turn Detection

### ServerVad
```python
class ServerVad(Model):
    type: Literal["server_vad"]
    threshold: Optional[float]  # 0.0-1.0
    prefix_padding_ms: Optional[int]
    silence_duration_ms: Optional[int]
    create_response: Optional[bool]
```

### AzureSemanticVad
```python
class AzureSemanticVad(Model):
    type: Literal["azure_semantic_vad"]
    # Uses semantic understanding for better turn detection
```

### AzureSemanticVadEn
```python
class AzureSemanticVadEn(Model):
    type: Literal["azure_semantic_vad_en"]
    eou_detection: Optional[EouDetection]
```

### EouDetection
```python
class EouDetection(Model):
    threshold_level: Optional[EouThresholdLevel]  # "low", "medium", "high", "default"
```

---

## Response Models

### Response
```python
class Response(Model):
    id: str
    object: Literal["realtime.response"]
    status: ResponseStatus
    status_details: Optional[ResponseStatusDetails]
    output: List[ResponseItem]
    usage: Optional[TokenUsage]
```

### ResponseCreateParams
```python
class ResponseCreateParams(Model):
    modalities: Optional[List[Modality]]
    instructions: Optional[str]
    voice: Optional[Voice]
    output_audio_format: Optional[OutputAudioFormat]
    tools: Optional[List[Tool]]
    tool_choice: Optional[ToolChoice]
    temperature: Optional[float]
    max_response_output_tokens: Optional[Union[int, Literal["inf"]]]
    conversation: Optional[Literal["auto", "none"]]
    input: Optional[List[ConversationRequestItem]]
```

### TokenUsage
```python
class TokenUsage(Model):
    total_tokens: int
    input_tokens: int
    output_tokens: int
    input_token_details: Optional[InputTokenDetails]
    output_token_details: Optional[OutputTokenDetails]
```

---

## Avatar Models

### AvatarConfig
```python
class AvatarConfig(Model):
    type: AvatarConfigTypes  # "video-avatar" or "photo-avatar"
    character: str
    style: Optional[str]
    output_protocol: Optional[AvatarOutputProtocol]  # "webrtc" or "websocket"
    background: Optional[Background]
    video_params: Optional[VideoParams]
```

### IceServer
```python
class IceServer(Model):
    urls: List[str]
    username: Optional[str]
    credential: Optional[str]
```

### Background
```python
class Background(Model):
    color: Optional[str]  # Hex color
    image_url: Optional[str]
```

### VideoParams
```python
class VideoParams(Model):
    resolution: Optional[VideoResolution]
    crop: Optional[VideoCrop]
```