Model Compose

Deploy production-ready AI services in minutes. One YAML file for agents, RAG pipelines, and MCP servers — run anywhere. Inspired by docker-compose.

LLM Mart 2 views 72 listing impressions

한국어 | 中文


model-compose

Deploy production-ready AI services in minutes.

One YAML file. Any model. Any protocol. Any runtime. Build chat APIs, RAG pipelines, autonomous agents, and MCP servers without writing application code — then deploy the same file anywhere, like docker-compose.

AI systems should not be locked into a single provider, runtime, or cloud. model-compose is built on four principles:

  • Composable — Models, agents, workflows, tools, memory, and protocols are interchangeable building blocks.
  • Portable — Define your AI system once, deploy anywhere without re-engineering.
  • Hybrid-First — Bridge cloud APIs and local models on your own terms.
  • Stream-Native — Data flows through workflows as it arrives — tokens, audio, frames, and events as first-class values.

Quick Start

Install with uv:

uv pip install model-compose

Or with pip:

pip install model-compose

Create model-compose.yml:

controller:
  adapter:
    type: http-server
    port: 8080
  webui:
    port: 8081

workflow:
  job:
    component: chatgpt
    input:
      prompt: ${input.prompt}

component:
  id: chatgpt
  type: http-client
  base_url: https://api.openai.com/v1
  action:
    path: /chat/completions
    method: POST
    headers:
      Authorization: Bearer ${env.OPENAI_API_KEY}
    body:
      model: gpt-4o
      messages:
        - role: user
          content: ${input.prompt}

Run it:

export OPENAI_API_KEY=your-key
model-compose up

That's it. You're serving GPT-4o at http://localhost:8080 with a web UI at http://localhost:8081. No application code. No framework boilerplate. Same file runs locally, in Docker, or in production.


What You Can Build

Here's what a single YAML file can serve today — just a few examples.

🤖 Autonomous Agents

Build a ReAct agent that plans, uses tools, and completes multi-step tasks — declaratively.

component:
  id: research-agent
  type: agent
  tools: [search-web, fetch-page]
  max_iteration_count: 10
  action:
    model:
      component: chatgpt
    system_prompt: You are a web research assistant.
    user_prompt: ${input.question}

See simple agents like a code reviewer, a RAG assistant, and a web researcher in agents/.

🔍 RAG Pipelines

Compose embedding, vector search, and generation into a single workflow — no glue code.

workflow:
  jobs:
    - id: embed
      component: embedder
      input: { text: ${input.query} }

    - id: retrieve
      component: knowledge
      action: search
      input: { vector: ${jobs.embed.output} }

    - id: answer
      component: chatgpt
      input:
        context: ${jobs.retrieve.output}
        question: ${input.query}

Native drivers ship for Chroma, Milvus, Qdrant, FAISS, Neo4j, ArangoDB, and Redis.

🌐 MCP Servers

Turn any workflow into an MCP server that Claude, ChatGPT, or Cursor can use — one line change.

controller:
  adapter:
    type: mcp-server   # ← was: http-server
    port: 8080

Full examples live in mcp-servers/, including a Slack bot MCP.

⚡ Streaming Multi-Modal Workflows

Stream tokens, audio chunks, and video frames end-to-end — first-class across every stage.

workflow:
  job:
    component: chatgpt
    output: ${output as sse-text}

component:
  id: chatgpt
  type: http-client
  action:
    body: { stream: true, ... }
    stream_format: json
    output: ${response[].choices[0].delta.content}

Real-time TTS, video-to-frames, and live chat examples live under data-streaming/ and showcase/.


From Development to Production

From the project's README.

Related servers

Universal MCP Server with advanced AI memory capabilities and semantic search.

30 views

MCP server for MikroTik routers: firewall, NAT, routing, DHCP, DNS, WireGuard and more via SSH.

28 views

Let Codex orchestrate external coding agents through their native harnesses.

28 views

Control plane MCP for scoped recon, triage, and bounded proofs.

27 views