Quillrag

Single-binary local RAG MCP server in Rust. MiniLM compiled in, hybrid search, zero deps.

LLM Mart 6 views 46 listing impressions

One file. Zero dependencies. Ready before your editor finishes loading.

A local RAG engine in a single static binary — MiniLM embeddings compiled inside, hybrid dense + BM25 retrieval, MCP-native. No Node, no Python, no model download on first query.

release platforms license

quillrag terminal demo: serve, index, search

Why quillrag

~20 ms to ready MCP handshake completes before the model even loads
Zero runtime deps no Node, no Python, no pip/npm, no model downloads — ever
Hybrid retrieval dense cosine ⊕ BM25 fused with Reciprocal Rank Fusion
Private by construction no network code path after installation
One file, three OSes ~105 MB (the model lives inside), CI-built for linux/macOS/Windows

Quick start

# 1. grab a prebuilt binary (or cargo install --path .)
gh release download --repo Ayush-yadav11/quillrag -p '*linux*'
tar xzf quillrag-x86_64-linux.tar.gz && chmod +x quillrag

# 2. point it at any folder of notes/docs/code
./quillrag index ~/notes          # incremental walk

# 3. ask it something
./quillrag search "how does backpropagation work"

Or wire it straight into Claude Desktop / Cursor and let the AI search your notes mid-conversation — config below.

$ ./quillrag serve --data-dir ~/.local/share/quillrag
2026-08-26 INFO quillrag 0.1.2 ready in 41ms      <- handshake-ready before the model loads

Why it's fast

Stage Cost
Binary start + MCP initialize ~20 ms (measured: store open + tool registration only)
First rag_search / rag_index call +~300 ms one-time (mmap safetensors, build BERT graph)
Subsequent searches ~25 ms per query (2-core CPU, small corpus)
Re-indexing unchanged corpus near-zero (FNV content hash skip)

The embedding model is lazy: the MCP handshake and rag_status never touch it, so editors see an instant server.

Install

Download a prebuilt archive from the latest release — Windows x86_64, macOS Apple Silicon, and Linux x86_64 are all built by CI on every version tag:

# linux/macOS example: fetch + extract the latest release
gh release download --repo Ayush-yadav11/quillrag -p '*linux*' | tar xz
chmod +x quillrag && ./quillrag --version

Linux requirement: v0.1.6 binaries are built on Ubuntu 24.04 runners and link glibc 2.39 (pidfd_spawnp/pidfd_getpid). On older distros (glibc < 2.39, e.g. Ubuntu 22.04), build from source with your local toolchain instead: cargo install --path ..

Or build from source:

cargo install --path .

Cross-compile targets used by CI: x86_64-unknown-linux-gnu, aarch64-apple-darwin, x86_64-pc-windows-msvc.

Wire it into your editor

Claude Desktop / Cursor / any MCP client:

{
  "mcpServers": {
    "quillrag": {
      "command": "/usr/local/bin/quillrag",
      "args": ["serve"],
      "env": { "QUILLRAG_DATA": "~/.local/share/quillrag" }
    }
  }
}

Or just run ./quillrag serve and point any stdio client at it.

Tools

From the project's README.