Mcp
Read any page past its anti-bot wall, crawl sites, search without an API key, solve captchas locally
- Transport
- Not stated
- Package
- —
- Registry id
- dev.ilien.svipall/mcp
No install snippet on purpose. A working MCP config is a command, its arguments and an environment block — the last two are where API keys live, so this catalogue never stores them and cannot publish them. Follow the link above for the authors' own instructions.
An MCP server and CLI, in Rust, that extracts web pages into Markdown,
crawls sites within configured limits, searches without an API key,
and attempts supported challenges, then says plainly which ones it could not.
Install · Use cases · Proof · Tools · Captcha · REST API · Compare · FAQ
Replayed from real runs on one machine. How it was recorded, and the raw captures.
Everything runs locally. There are no third-party API keys, no paid captcha services and no telemetry. Web requests reach the sites you visit, and results reach the agent or client you connect.
Svipall fetches and renders pages locally, extracts their content, and reports detected challenges and extraction-quality labels. Some sites still block it, and an apparently successful response can contain incomplete records or a page shell. The published comparisons therefore audit useful content separately from HTTP status and the tool's own verdict.
Why Svipall
| What goes wrong | Svipall |
|---|---|
Your agent reads a "checking your browser" screen and summarises it as the article. It was a 200, so nothing flagged it |
Twelve wall kinds, each naming the move it implies. Detected blocks carry an explicit verdict; classification is heuristic details |
| You crawl 5,000 pages and can't tell which are worth keeping | Assessed pages carry quality and duplicate observations, and quality labels never discard a page details |
| One page = 300,000 tokens of raw HTML. The fixes are four manual jobs you now own | Clean Markdown by default; opt into tables as rows, out_file to disk, or capture of the site's own JSON API details |
| You want to attempt a supported captcha without a paid solver | Fifteen widget families and eleven answer modalities, all local, optional vision models depending on the build, and a human dashboard for unresolved challenges details |
It records successful visits, failures, incomplete extraction and rejected changes. Historical benchmark logs and the current comparison use different scoring rules and configurations; the results section distinguishes them.
From the project's README.