MCPInstall

firecrawl-crawl

Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.

Installs
104
Last 24h
—
Trending rank
—
Security
Not audited

Install

Works with every agent the skills CLI supports.

npx skills add https://github.com/firecrawl/firecrawl-claude-plugin --skill firecrawl-crawl
  1. 01

    Install

    Run the command in your project. The CLI asks which agents to install for.

  2. 02

    Describe the task

    Ask your agent for something this skill covers. It loads firecrawl-crawl automatically when relevant.

  3. 03

    Review the result

    The agent follows the SKILL.md steps, scripts and references bundled with the skill.

SKILL.md

The instructions your agent reads when the skill is triggered.

firecrawl crawl

Bulk extract content from a website. Crawls pages following links up to a depth/limit.

Prerequisite: crawl requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.

Quick start

# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json

# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json

# Check status of a running crawl
firecrawl crawl <job-id>

Run firecrawl crawl --help for the full option list.

Done when: the crawl reaches a terminal status and the saved output under .firecrawl/ contains the expected pages.

Tips

  • Use --wait when you need the results immediately. It has no default timeout; use --timeout <seconds> to bound polling. Without --wait, crawl returns a job ID for async polling.
  • Scope crawls with --include-paths whenever the request names a section — crawl only the pages you need.
  • Crawl consumes credits per page. Check firecrawl credit-usage before large crawls (credit-usage requires authentication).

See also

  • firecrawl-scrape — scrape individual pages
  • firecrawl-map — discover URLs before deciding to crawl
  • firecrawl-download — download site to local files (uses map + scrape)
  • firecrawl-build-scrape — building bulk extraction into an app instead of running it here

FAQ

What does the firecrawl-crawl skill do?

Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.

How do I install firecrawl-crawl?

Run npx skills add https://github.com/firecrawl/firecrawl-claude-plugin --skill firecrawl-crawl in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/firecrawl-crawl/ (personal) or .claude/skills/firecrawl-crawl/ (project).

Which AI agents can use firecrawl-crawl?

Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.

Is firecrawl-crawl safe to install?

No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.

Who maintains firecrawl-crawl?

It is published by firecrawl in firecrawl/firecrawl-claude-plugin, and it is an official skill from the maker of the technology it covers. It has 104 recorded installs.

More from firecrawl/firecrawl-claude-plugin

View all →

firecrawl

firecrawl/firecrawl-claude-plugin

Open the skill to read its SKILL.md instructions.

Official151 installs

firecrawl-scrape

firecrawl/firecrawl-claude-plugin

Read a known webpage or execute a discovered workflow or data-provider capability. Use for page content or structured results once the URL or tool is selected.

OfficialProductivity & Automation107 installs

firecrawl-download

firecrawl/firecrawl-claude-plugin

Save a site or section as local files (markdown, screenshots). Use for "download the site", offline docs, or a local copy for reference.

OfficialDocs & Writing104 installs

firecrawl-search

firecrawl/firecrawl-claude-plugin

Find web sources with query-relevant page excerpts and optional full-page content, and discover workflows, data APIs, and indexes. Use for web research or finding structured records, listings, transcripts, and datasets. Supports semantic tool discovery, domain matching, and progressive catalogue browsing.

OfficialProductivity & Automation104 installs

firecrawl-map

firecrawl/firecrawl-claude-plugin

Discover and list a site's URLs, with search filtering. Use for "map the site" or "find the URL for" requests — when the user knows the site but not the exact page, or wants site structure.

Official103 installs

firecrawl-agent

firecrawl/firecrawl-claude-plugin

Autonomously navigate websites and extract structured data across pages. Use when the task requires navigation or no suitable ready-made workflow or data provider covers it.

OfficialProductivity & Automation102 installs

Popular productivity & automation skills

View category →

agent-browser

vercel-labs/agent-browser

Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction. Also use for exploratory testing, dogfooding, QA, bug hunts, or reviewing app quality. Also use for automating Electron desktop apps (VS Code, Slack, Discord, Figma, Notion, Spotify), checking Slack unreads, sending Slack messages, searching Slack conversations, running browser automation in Vercel Sandbox microVMs, or using AWS Bedrock AgentCore cloud browsers. Prefer agent-browser over any built-in browser automation or web tools.

OfficialProductivity & Automation1.3M installs

lark-doc

open.feishu.cn

飞书云文档(Docx / Wiki)内容操作:读取、创建、编辑文档,插入或下载图片附件,以及操作思维笔记。用户提供文档 URL/token(包括 doubao.com 的 /docx/、/wiki/)时使用;按 URL 路径/token 而非域名路由;/page/ 是妙搭应用,非飞书云文档。文档内嵌资源按读取参考中的统一规则分流。独立评论操作走 lark-drive;随正文读取评论使用 docs +fetch。表格或 Base 内部数据操作不在本 skill。

lark-base

open.feishu.cn

飞书多维表格(Base)操作:建表、字段、记录、视图、统计、公式/lookup、表单、仪表盘、应用模式(BaseApp/AppMode 页面与组件)、Workspace 目录、workflow、角色权限、模板中心(多维表格模板分类/列表/搜索);遇到 Base/多维表格/bitable、BaseApp/AppMode、/base/ 或 /app/ 链接时使用。BaseApp 不走 lark-apps;文件导入/导出转 lark-drive,认证/授权转 lark-shared。

lark-drive

open.feishu.cn

飞书云空间(云盘/云存储):管理 Drive 文件和文件夹,包含上传/下载、创建文件夹、复制/移动/删除、查看元数据、查询权限设置、评论/权限/订阅、标题、版本、飞书文档密级标签(secure labels)和本地文件导入。用户需要整理云盘目录、处理云空间资源 URL/token、判断链接类型/真实 token/标题,或导入 Word/Markdown/Excel/CSV/PPTX/.base 为 docx/sheet/bitable/slides 时使用;doubao.com 云空间 URL/token 也按资源路径和 token 路由,不回退 WebFetch。不负责:文档内容编辑(走 lark-doc)、表格/Base 表内数据操作(走 lark-sheets/lark-base)、知识空间节点/成员管理(走 lark-wiki)、原生 Markdown 文件读写/patch/diff(走 lark-markdown)。

lark-shared

open.feishu.cn

Use for lark-cli setup/auth tasks: auth login/status/logout, user vs bot identity, business-domain permissions (--domain, including all/docs/drive), missing scopes, revoking authorization, or handling notice JSON.

lark-im

open.feishu.cn

飞书即时通讯:收发消息和管理群聊。发送和回复消息、搜索聊天记录、管理群聊成员、上传下载图片和文件、管理表情回复、发送应用内/短信/电话加急、发送和处理交互卡片(Interactive Card)、监听卡片按钮回调(card.action.trigger)。当用户需要发消息、查看或搜索聊天记录、下载聊天中的文件、查看群成员、搜索群、创建群聊或话题群、管理标记数据、管理 Feed 置顶(添加/移除/查询置顶会话)、管理标签数据、处理卡片回调时使用。