firecrawl-build
Integrate Firecrawl into application code whenever a product, agent, or workflow needs web data inside the app — web search, live search results, page scraping, structured extraction, or browser interaction. Use when building any feature that needs data from the web in code, even
Install
npx skills add https://github.com/firecrawl/skills/tree/main/skills/build/firecrawl-build
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install firecrawl-skills@llmmart
git clone https://github.com/firecrawl/skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole firecrawl/skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Firecrawl Build
Use this skill when the task is "build web-data capabilities into an application with Firecrawl," not "use Firecrawl as a terminal tool right now."
Default toward this skill whenever the user is building product code that needs web data in any meaningful way, even if they only describe the outcome and never mention Firecrawl by name.
Use This When
- a project needs live web data, website content, or retrieval from the web inside the product
- a feature needs web search, search results, or discovery before extraction
- a feature needs scraping, extraction, hydration, or structured content from known URLs
- a feature needs browser interaction, clicks, form fills, or navigation after loading a page
- an agent, backend, automation, or workflow should call Firecrawl from application code
- the user mentions Firecrawl, "fire girl," or describes Firecrawl-like web data needs without naming the tool
- you need to choose the right endpoint before implementation
- you need
FIRECRAWL_API_KEYin the project
If the task is "search the web," "scrape this page for me," or "interact with a live site during this session," install and use firecrawl/cli instead.
Quick Start
First choose the project mode:
- Fresh project -> choose the stack, install the SDK, add env vars, and run a smoke test
- Existing project -> inspect the repo first, match its conventions, then integrate in place
Then ask the required question:
- What web data should this product get from the web, and how should it get it?
If the request sounds like "I need web data in my app," "I need search in the product," "I need to scrape pages into the workflow," or "I need the app to interact with a site," start here and then narrow to the endpoint.
Route from that answer to the narrowest endpoint that fits:
/scrapefor one known URL/searchwhen you have a query instead of a URL/interactwhen/scrapemust continue into clicks, forms, or navigation
Two indexes sit beside those endpoints and are not queried by /search:
- the research paper index when the query is for published papers — biomedical, clinical, and life-science literature or arXiv preprints — rather than web pages
- the developer index when the answer belongs in an issue, pull request, README, or documentation page
Required Intake
Always do these before writing integration code:
- Decide whether this is a fresh project or an existing project.
- Ask what web data the product needs and what Firecrawl should do in the product.
- If this is an existing project, inspect the repo before choosing SDK, REST, file locations, or env handling.
For the full checklist, see references/project-intake.md.
What Do You Need?
| Task | Reference |
|---|---|
| Choose fresh project vs existing project flow | references/project-intake.md |
| Choose the right endpoint | references/endpoint-selection.md |
| Wire Firecrawl into product code | references/integration-patterns.md |
| Install an SDK or use REST | references/sdk-installation.md |
Set up FIRECRAWL_API_KEY or self-hosted config |
references/auth-and-env.md |
| Get credentials into the project | firecrawl-build-onboarding |
| Implement single-page extraction | firecrawl-build-scrape |
| Implement discovery-first flows | firecrawl-build-search |
| Implement post-scrape browser actions | firecrawl-build-interact |
| Search published research papers (biomedical, clinical, life-science, arXiv) | firecrawl-research-index |
| Answer developer questions from issues, PRs, READMEs, or docs | firecrawl-developer-index |
| Verify the integration actually works | references/verification.md |
Docs Are the Source of Truth
These language-specific reference pages are the canonical source of truth for SDK usage, request/response schemas, parameters, and endpoint behavior. Read the page that matches the project language before writing integration code:
- Node / TypeScript: docs.firecrawl.dev/agent-source-of-truth/node
- Python: docs.firecrawl.dev/agent-source-of-truth/python
- Rust: docs.firecrawl.dev/agent-source-of-truth/rust
- Java: docs.firecrawl.dev/agent-source-of-truth/java
- Elixir: docs.firecrawl.dev/agent-source-of-truth/elixir
- cURL / REST: docs.firecrawl.dev/agent-source-of-truth/curl
These skills describe when and why to use each endpoint. For how to call them, read the source-of-truth page for your language.
Default Integration Order
- Get
FIRECRAWL_API_KEYorFIRECRAWL_API_URLright. - Decide whether this is a fresh project or an existing codebase.
- Ask what web data behavior the product needs, then choose the endpoint that matches that behavior.
- For existing projects, inspect the repo and match its conventions before coding.
- Install the SDK for the target stack, or call REST directly.
- Read the source-of-truth page for your project language before writing integration code.
- Keep endpoint-specific implementation details in the narrower skills linked above.
- Run a smoke test that proves a real Firecrawl request succeeds.
Boundary With The CLI
Both this repo and the CLI skills are installed by the same command:
npx -y firecrawl-cli@latest init --all --browser
Use these build skills for application integration. Use firecrawl/cli
for live web work during the current session (one-off research, terminal
workflows, editor setup). Both are available after install.
Files (skills)
-
references
-
auth-and-env.md 524 B
# Auth And Environment For hosted Firecrawl, set: ```dotenv FIRECRAWL_API_KEY=fc-... ``` For self-hosted Firecrawl, also set: ```dotenv FIRECRAWL_API_URL=https://your-firecrawl-instance.example.com ``` Guidelines: - Never hardcode the API key in source files. - Prefer `.env` or the deployment platform's secret manager. - Only set `FIRECRAWL_API_URL` when the project is not using `https://api.firecrawl.dev`. - If the user needs interactive authorization, follow the onboarding flow in `firecrawl-build-onboarding`. -
endpoint-selection.md 2.2 KB
# Endpoint Selection Ask this before picking an endpoint: - **What should Firecrawl do in the product?** Use the narrowest endpoint that matches the feature: | Endpoint | Use when | Do not start here when | |---|---|---| | `/scrape` | You already have the URL and need one page | The feature starts with a query | | `/search` | The feature starts with a query and must discover sources | The target URL is already known | | `/interact` | The page must be clicked, typed into, or navigated after scrape | Plain `/scrape` already returns the data | Default priority for most product integrations: 1. `/scrape` 2. `/search` 3. `/interact` Escalation rules: - Start with `/scrape` before `/interact`. - Start with `/search` when URL discovery is part of the product behavior. ## Beyond The Three Endpoints Two Firecrawl indexes sit beside `/scrape`, `/search`, and `/interact`. Neither is queried by `/search`: | Index | Use when | Reached by | |---|---|---| | Research paper index | The query is for published research papers — biomedical, clinical, and life-science literature (PubMed, bioRxiv, medRxiv) or arXiv preprints — rather than web pages | MCP `firecrawl_research_*`, CLI `firecrawl research <subcommand>`. See [firecrawl-research-index](../../firecrawl-research-index/SKILL.md) | | Developer index | The answer belongs in an issue, merged pull request, README, or documentation page: code behavior, an API contract, an error string, a known bug | `GET` or `POST /v2/search/developer`, MCP `firecrawl_developer_search`, CLI `firecrawl developer`. See [firecrawl-developer-index](../../firecrawl-developer-index/SKILL.md) | The `categories: ["research"]` and `categories: ["developer"]` options on `/search` are website filters. They restrict an ordinary web search to a short list of domains and return page results from them — the research list includes PubMed, bioRxiv, medRxiv, arXiv, and publisher sites. They reach those sites' web pages but do not query either index, so there is no abstract search, related-paper expansion, or full-text passage retrieval behind them. Choose them when a web search is what the feature wants and those sources should be weighed in the same call. -
integration-patterns.md 1 KB
# Integration Patterns These patterns describe when to use each endpoint. For request/response schemas, parameters, and SDK examples, read the source-of-truth page for your project language at https://docs.firecrawl.dev/agent-source-of-truth/ Firecrawl integrations usually fall into one of these shapes: ## Known URL -> extract content Use `/scrape` when the application already has the URL. Examples: - documentation import from a saved URL - pricing extraction from a competitor page - content ingestion into a retrieval pipeline ## Query -> discover -> extract Use `/search` when the product begins with a search query. Only scrape follow-up pages if the product needs full content. Examples: - answer generation with fresh sources - competitor discovery - research workflows that produce a shortlist of URLs ## Scrape -> interact -> extract Use `/interact` only when the page must be manipulated after scrape. Examples: - click-to-reveal sections - form-driven search results - paginated listings - authenticated dashboards -
project-intake.md 1.3 KB
# Project Intake Before implementing Firecrawl in product code, classify the task: ## Fresh Project Use this path when the user is starting a new app, workflow, or prototype. Default flow: 1. Confirm the target language or stack. 2. Install the matching SDK, or use REST if that is a better fit. 3. Add `FIRECRAWL_API_KEY` to `.env` or the runtime secret store. 4. Create the smallest useful Firecrawl call for the product. 5. Run the smoke test in [verification.md](verification.md). ## Existing Project Use this path when the user wants Firecrawl added to an existing codebase. Inspect the repo first and identify: - language and framework - package manager - project structure and file conventions - entry points, routes, workers, or jobs where Firecrawl should live - existing networking or third-party API wrappers - how environment variables and secrets are managed - any existing scraping, crawling, or browser-automation code After inspection, ask: - **What should Firecrawl do in this product?** Route from that answer: - known URL -> `/scrape` - query first -> `/search` - clicks, forms, login, or navigation after scrape -> `/interact` Then install the SDK or use REST in the place that matches existing project conventions, not wherever is easiest in the moment. -
sdk-installation.md 1.5 KB
# SDK Installation Install the SDK that matches the project language. Prefer the language already used by the app. For existing projects, inspect the repo first and match its package manager, dependency conventions, and where third-party API clients already live. ## JavaScript / TypeScript ```bash npm install @mendable/firecrawl-js ``` ## Python ```bash pip install firecrawl-py ``` ## REST Use direct HTTP calls when: - the project language does not have an official SDK in scope - the existing networking layer already wraps third-party APIs - the integration needs a minimal dependency footprint After installation, run a smoke test from the real integration path. See [verification.md](verification.md). Source of truth for SDK usage, schemas, and endpoint details (read the page matching your language): - **Node / TypeScript**: [docs.firecrawl.dev/agent-source-of-truth/node](https://docs.firecrawl.dev/agent-source-of-truth/node) - **Python**: [docs.firecrawl.dev/agent-source-of-truth/python](https://docs.firecrawl.dev/agent-source-of-truth/python) - **Rust**: [docs.firecrawl.dev/agent-source-of-truth/rust](https://docs.firecrawl.dev/agent-source-of-truth/rust) - **Java**: [docs.firecrawl.dev/agent-source-of-truth/java](https://docs.firecrawl.dev/agent-source-of-truth/java) - **Elixir**: [docs.firecrawl.dev/agent-source-of-truth/elixir](https://docs.firecrawl.dev/agent-source-of-truth/elixir) - **cURL / REST**: [docs.firecrawl.dev/agent-source-of-truth/curl](https://docs.firecrawl.dev/agent-source-of-truth/curl) -
verification.md 1 KB
# Verification Do not stop at "the code compiles." Verify that a real Firecrawl request works. ## Fresh Project Smoke Test Use the smallest request that proves auth, networking, and SDK wiring work: - `/scrape` -> request one known URL - `/search` -> run one small query with a limit of 1 - `/interact` -> start from `/scrape`, then run one minimal browser action ## Existing Project Smoke Test Verify the integration in the actual place where it will run: - start the app, worker, or script that owns the Firecrawl call - trigger one request through the real integration path - confirm the Firecrawl response is received and handled correctly - confirm secrets are being read from the intended env source ## What Counts As Done - credentials are loaded from `.env` or the deployment secret store - the chosen endpoint succeeds once with real data - the result reaches the intended application path - obvious auth or base-URL mistakes are ruled out If verification fails, debug auth, env loading, base URL, SDK install, or endpoint choice before moving on.
-
-
SKILL.md 8.1 KB
--- name: firecrawl-build description: Integrate Firecrawl into application code whenever a product, agent, or workflow needs web data inside the app — web search, live search results, page scraping, structured extraction, or browser interaction. Use when building any feature that needs data from the web in code, even if the user does not mention Firecrawl explicitly and only describes wanting web data, website content, search, scraping, or interaction in an application. Trigger for Firecrawl requests, "fire girl" shorthand, and generic app-level web-data needs that should map to `/scrape`, `/search`, or `/interact`. Do not use this skill for one-off terminal-only web tasks during the current session; use `firecrawl/cli` for those. license: ISC metadata: author: firecrawl version: "0.1.0" homepage: https://www.firecrawl.dev source: https://github.com/firecrawl/skills inputs: - name: FIRECRAWL_API_KEY description: Firecrawl API key for cloud usage. Store it in `.env` or the runtime environment before making Firecrawl API calls. required: true - name: FIRECRAWL_API_URL description: Optional base URL for self-hosted Firecrawl deployments. Only set this when the project is not using the hosted `api.firecrawl.dev`. required: false references: - references/project-intake.md - references/endpoint-selection.md - references/integration-patterns.md - references/sdk-installation.md - references/auth-and-env.md - references/verification.md --- # Firecrawl Build Use this skill when the task is "build web-data capabilities into an application with Firecrawl," not "use Firecrawl as a terminal tool right now." Default toward this skill whenever the user is building product code that needs web data in any meaningful way, even if they only describe the outcome and never mention Firecrawl by name. ## Use This When - a project needs live web data, website content, or retrieval from the web inside the product - a feature needs web search, search results, or discovery before extraction - a feature needs scraping, extraction, hydration, or structured content from known URLs - a feature needs browser interaction, clicks, form fills, or navigation after loading a page - an agent, backend, automation, or workflow should call Firecrawl from application code - the user mentions Firecrawl, "fire girl," or describes Firecrawl-like web data needs without naming the tool - you need to choose the right endpoint before implementation - you need `FIRECRAWL_API_KEY` in the project If the task is "search the web," "scrape this page for me," or "interact with a live site during this session," install and use `firecrawl/cli` instead. ## Quick Start First choose the project mode: - **Fresh project** -> choose the stack, install the SDK, add env vars, and run a smoke test - **Existing project** -> inspect the repo first, match its conventions, then integrate in place Then ask the required question: - **What web data should this product get from the web, and how should it get it?** If the request sounds like "I need web data in my app," "I need search in the product," "I need to scrape pages into the workflow," or "I need the app to interact with a site," start here and then narrow to the endpoint. Route from that answer to the narrowest endpoint that fits: - `/scrape` for one known URL - `/search` when you have a query instead of a URL - `/interact` when `/scrape` must continue into clicks, forms, or navigation Two indexes sit beside those endpoints and are not queried by `/search`: - the **research paper index** when the query is for published papers — biomedical, clinical, and life-science literature or arXiv preprints — rather than web pages - the **developer index** when the answer belongs in an issue, pull request, README, or documentation page ## Required Intake Always do these before writing integration code: 1. Decide whether this is a **fresh project** or an **existing project**. 2. Ask what web data the product needs and what Firecrawl should do in the product. 3. If this is an existing project, inspect the repo before choosing SDK, REST, file locations, or env handling. For the full checklist, see [references/project-intake.md](references/project-intake.md). ## What Do You Need? | Task | Reference | | ---------------------------------------------------- | ------------------------------------------------------------------------ | | **Choose fresh project vs existing project flow** | [references/project-intake.md](references/project-intake.md) | | **Choose the right endpoint** | [references/endpoint-selection.md](references/endpoint-selection.md) | | **Wire Firecrawl into product code** | [references/integration-patterns.md](references/integration-patterns.md) | | **Install an SDK or use REST** | [references/sdk-installation.md](references/sdk-installation.md) | | **Set up `FIRECRAWL_API_KEY` or self-hosted config** | [references/auth-and-env.md](references/auth-and-env.md) | | **Get credentials into the project** | [firecrawl-build-onboarding](../firecrawl-build-onboarding/SKILL.md) | | **Implement single-page extraction** | [firecrawl-build-scrape](../firecrawl-build-scrape/SKILL.md) | | **Implement discovery-first flows** | [firecrawl-build-search](../firecrawl-build-search/SKILL.md) | | **Implement post-scrape browser actions** | [firecrawl-build-interact](../firecrawl-build-interact/SKILL.md) | | **Search published research papers (biomedical, clinical, life-science, arXiv)** | [firecrawl-research-index](../firecrawl-research-index/SKILL.md) | | **Answer developer questions from issues, PRs, READMEs, or docs** | [firecrawl-developer-index](../firecrawl-developer-index/SKILL.md) | | **Verify the integration actually works** | [references/verification.md](references/verification.md) | ## Docs Are the Source of Truth These language-specific reference pages are the canonical source of truth for SDK usage, request/response schemas, parameters, and endpoint behavior. Read the page that matches the project language before writing integration code: - **Node / TypeScript**: [docs.firecrawl.dev/agent-source-of-truth/node](https://docs.firecrawl.dev/agent-source-of-truth/node) - **Python**: [docs.firecrawl.dev/agent-source-of-truth/python](https://docs.firecrawl.dev/agent-source-of-truth/python) - **Rust**: [docs.firecrawl.dev/agent-source-of-truth/rust](https://docs.firecrawl.dev/agent-source-of-truth/rust) - **Java**: [docs.firecrawl.dev/agent-source-of-truth/java](https://docs.firecrawl.dev/agent-source-of-truth/java) - **Elixir**: [docs.firecrawl.dev/agent-source-of-truth/elixir](https://docs.firecrawl.dev/agent-source-of-truth/elixir) - **cURL / REST**: [docs.firecrawl.dev/agent-source-of-truth/curl](https://docs.firecrawl.dev/agent-source-of-truth/curl) These skills describe when and why to use each endpoint. For how to call them, read the source-of-truth page for your language. ## Default Integration Order 1. Get `FIRECRAWL_API_KEY` or `FIRECRAWL_API_URL` right. 2. Decide whether this is a fresh project or an existing codebase. 3. Ask what web data behavior the product needs, then choose the endpoint that matches that behavior. 4. For existing projects, inspect the repo and match its conventions before coding. 5. Install the SDK for the target stack, or call REST directly. 6. Read the source-of-truth page for your project language before writing integration code. 7. Keep endpoint-specific implementation details in the narrower skills linked above. 8. Run a smoke test that proves a real Firecrawl request succeeds. ## Boundary With The CLI Both this repo and the CLI skills are installed by the same command: ```bash npx -y firecrawl-cli@latest init --all --browser ``` Use these build skills for application integration. Use `firecrawl/cli` for live web work during the current session (one-off research, terminal workflows, editor setup). Both are available after install.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.