Macos Vision Mcp

Local OCR & screen understanding for agents: read any window, find what to click. No uploads.

LLM Mart 6 views 64 listing impressions

Local, private, offline OCR and UI testing for any MCP client — no API keys, no uploads. Cut document token costs by ~97%, and let an agent see and click your Mac's UI without a single screenshot leaving the machine.

npm version npm downloads GitHub stars License: MIT macOS 13.0+ No API Key Offline Glama

Pre-extracts text and image data locally before your AI ever sees it — cutting token usage by ~97% on real documents and returning structured paragraphs, lines, and bounding boxes so the model can reconstruct the document into Markdown, HTML, DOCX, or any other format. Files never leave your Mac: no cloud API, no API keys, no network requests.

How the ~97% is measured: a 44-page scanned PDF sent as page images costs ~73,500 tokens; the same file run through analyze_document returns ~2,400 tokens of extracted text and structure (raw page-image tokens vs. extracted-text tokens). Your numbers vary with page density and tokenizer — treat 97% as the order of magnitude, not a guarantee.

Contents: Quick Start · What you get · What agents use this for · UI testing · Why it's different · Available Tools · Usage · Example workflows · Configuration · Privacy layer

What you get

  • OCR for images and PDFs (JPG, PNG, HEIC, TIFF, multi-page PDF) via Apple Vision Framework.
  • ~97% token reduction: a 44-page PDF costs ~2,400 tokens instead of ~73,500.
  • Reading-order paragraphs + raw text blocks with bounding boxes — rich structure for the model to reconstruct the document into any output format (Markdown, HTML, DOCX, JSON), not a lossy plain-text dump.
  • Face detection, barcode/QR reading, and image classification — all on-device.
  • Full document pipeline: OCR + faces + barcodes + rectangles in a single tool call.
  • Works with Claude Code, Claude Desktop, and Cursor — any MCP-compatible client.
  • No files uploaded to any server — processing stays entirely on your Mac.
  • UI testing for agents: screenshot a window locally, find an element by its visible text, get back click coordinates, and assert what's on screen — all without uploading the screenshot.
  • 100% offline after npm install — powered by Apple Vision Framework, same engine as Live Text in Photos.app.

❌ Without / ✅ With

❌ Without macos-vision-mcp:

  • Sending a 44-page PDF costs ~73,500 tokens
  • Every image, invoice, or contract goes through a cloud API
  • Sensitive documents leave your machine on every request

✅ With macos-vision-mcp:

  • Local Apple Vision pre-extracts text before Claude ever sees it
  • ~2,400 tokens for the same 44-page PDF — 97% fewer
  • Files never leave your Mac

What agents use this for

From the project's README.