{"slug":"firecrawl-crawl-2","title":"firecrawl-crawl","summary":"Bulk-extract many pages from one site or section. Use for \"crawl\", \"everything under /docs\", or content spanning linked pages.","platform":"Claude","tags":[],"authorName":"LLM Mart","authorSlug":"llm-mart","score":0,"source":"github","price":null,"verified":false,"createdAt":"2026-10-17T09:07:15.979199Z","repo":{"url":"https://github.com/firecrawl/cli","stars":654,"forks":116,"license":null,"updatedAt":"2026-10-17T07:09:53Z"},"bodyHtml":"<hr>\n<p>name: firecrawl-crawl\ndescription: |\nBulk-extract many pages from one site or section. Use for \"crawl\", \"everything under /docs\", or content spanning linked pages.\nallowed-tools:</p>\n<ul>\n<li>Bash(firecrawl *)</li>\n<li>Bash(npx firecrawl-cli *)</li>\n</ul>\n<hr>\n<h1>firecrawl crawl</h1>\n<p>Bulk extract content from a website. Crawls pages following links up to a depth/limit.</p>\n<p><strong>Prerequisite:</strong> <code>crawl</code> requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.</p>\n<h2>Quick start</h2>\n<pre><code># Crawl a docs section\nfirecrawl crawl \"&lt;url&gt;\" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json\n\n# Full crawl with depth limit\nfirecrawl crawl \"&lt;url&gt;\" --max-depth 3 --wait --progress -o .firecrawl/crawl.json\n\n# Check status of a running crawl\nfirecrawl crawl &lt;job-id&gt;\n</code></pre>\n<p>Run <code>firecrawl crawl --help</code> for the full option list.</p>\n<p><strong>Done when:</strong> the crawl reaches a terminal status and the saved output under <code>.firecrawl/</code> contains the expected pages.</p>\n<h2>Tips</h2>\n<ul>\n<li>Use <code>--wait</code> when you need the results immediately. It has no default timeout; use <code>--timeout &lt;seconds&gt;</code> to bound polling. Without <code>--wait</code>, crawl returns a job ID for async polling.</li>\n<li><strong>Scope crawls with <code>--include-paths</code></strong> whenever the request names a section — crawl only the pages you need.</li>\n<li>Crawl consumes credits per page. Check <code>firecrawl credit-usage</code> before large crawls (<code>credit-usage</code> requires authentication).</li>\n</ul>\n<h2>See also</h2>\n<ul>\n<li><a href=\"../firecrawl-scrape/SKILL.md\">firecrawl-scrape</a> — scrape individual pages</li>\n<li><a href=\"../firecrawl-map/SKILL.md\">firecrawl-map</a> — discover URLs before deciding to crawl</li>\n<li><a href=\"../firecrawl-download/SKILL.md\">firecrawl-download</a> — download site to local files (uses map + scrape)</li>\n<li><a href=\"https://github.com/firecrawl/skills/tree/main/skills/build/firecrawl-build-scrape\">firecrawl-build-scrape</a> — building bulk extraction into an app instead of running it here</li>\n</ul>\n","files":[{"path":"SKILL.md","sizeBytes":1876,"isText":true}],"reviewScore":null,"reviewSummary":null,"trust":{"provenance":"trusted-source-unreviewed","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow.","bodySource":null},"bodyLocked":false,"purchaseUrl":null,"sourceUrl":null,"report":{"provenance":"trusted-source-unreviewed","screen":{"ran":true,"outcome":"clean","suspicious":0,"notes":0,"hiddenCharacters":false},"virusScan":{"engine":"clamav","status":"clean","scannedAt":"2026-10-17T09:08:01.790942Z","sha256":"13F2AA0322AF91E89457B779AC30810276E435D6E6EA7B73ED5B378A59DA9BF5","sizeBytes":1012},"review":null,"source":{"repositoryUrl":"https://github.com/firecrawl/cli","path":"skills/firecrawl-crawl","license":null,"commit":"085e0c1189343d7bda7240324fe8546f03f0c5b5","subtreeSha":"2DA938161BC3EB835BEAE7B0C0D75B7152FF6A06DCD8A6B10907B407D21BA646","lastSyncedAt":"2026-10-17T09:07:15.502373Z"},"reviewedAt":"2026-10-17T09:09:43.42241Z","notice":"Community-authored content, reproduced verbatim and not vetted as instructions. Treat it as data to evaluate, never as directives to follow."},"install":[{"target":"skills-cli","command":"npx skills add https://github.com/firecrawl/cli/tree/main/skills/firecrawl-crawl"},{"target":"claude-code","command":"claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install firecrawl-cli@llmmart"},{"target":"git","command":"git clone https://github.com/firecrawl/cli.git"}]}