Macos Vision Mcp
Local OCR & screen understanding for agents: read any window, find what to click. No uploads.
Local, private, offline OCR and UI testing for any MCP client — no API keys, no uploads. Cut document token costs by ~97%, and let an agent see and click your Mac's UI without a single screenshot leaving the machine.
Pre-extracts text and image data locally before your AI ever sees it — cutting token usage by ~97% on real documents and returning structured paragraphs, lines, and bounding boxes so the model can reconstruct the document into Markdown, HTML, DOCX, or any other format. Files never leave your Mac: no cloud API, no API keys, no network requests.
How the ~97% is measured: a 44-page scanned PDF sent as page images costs ~73,500 tokens; the same file run through
analyze_documentreturns ~2,400 tokens of extracted text and structure (raw page-image tokens vs. extracted-text tokens). Your numbers vary with page density and tokenizer — treat 97% as the order of magnitude, not a guarantee.
Contents: Quick Start · What you get · What agents use this for · UI testing · Why it's different · Available Tools · Usage · Example workflows · Configuration · Privacy layer
What you get
- OCR for images and PDFs (JPG, PNG, HEIC, TIFF, multi-page PDF) via Apple Vision Framework.
- ~97% token reduction: a 44-page PDF costs ~2,400 tokens instead of ~73,500.
- Reading-order paragraphs + raw text blocks with bounding boxes — rich structure for the model to reconstruct the document into any output format (Markdown, HTML, DOCX, JSON), not a lossy plain-text dump.
- Face detection, barcode/QR reading, and image classification — all on-device.
- Full document pipeline: OCR + faces + barcodes + rectangles in a single tool call.
- Works with Claude Code, Claude Desktop, and Cursor — any MCP-compatible client.
- No files uploaded to any server — processing stays entirely on your Mac.
- UI testing for agents: screenshot a window locally, find an element by its visible text, get back click coordinates, and assert what's on screen — all without uploading the screenshot.
- 100% offline after
npm install— powered by Apple Vision Framework, same engine as Live Text in Photos.app.
❌ Without / ✅ With
❌ Without macos-vision-mcp:
- Sending a 44-page PDF costs ~73,500 tokens
- Every image, invoice, or contract goes through a cloud API
- Sensitive documents leave your machine on every request
✅ With macos-vision-mcp:
- Local Apple Vision pre-extracts text before Claude ever sees it
- ~2,400 tokens for the same 44-page PDF — 97% fewer
- Files never leave your Mac
What agents use this for
From the project's README.