firecrawl-knowledge-base
Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources.
- Installs
- 32.6K
- Last 24h
- +26
- Trending rank
- #4763
- Security
- Warn
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/firecrawl/firecrawl-workflows --skill firecrawl-knowledge-base- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads firecrawl-knowledge-base automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
Security audits
Independent scans of the skill's instructions and bundled files.
SKILL.md
The instructions your agent reads when the skill is triggered.
Firecrawl Knowledge Base
Use this to turn URLs or topics into organized LLM-ready content.
Onboarding Interview
Infer the source, goal, depth, and output location from context. If the source and goal are clear, proceed immediately.
Ask at most 1-3 concise questions only if blocked, such as the source URL/topic, whether the output is reference/RAG/training/docs, or training format if training is requested.
Firecrawl Collection Plan
Use Firecrawl map for documentation sites, search for topic-based corpora, scrape pages into markdown, and preserve code examples and tables.
For files, follow the Firecrawl download-style convention:
.firecrawl/
<hostname>/
<path>/
index.md
Parallel Work
If appropriate, use sub-agents or equivalent parallel task runners:
- one docs section per researcher
- official docs, tutorials, community discussions, and references by source type
- source scraping vs chunk generation vs manifest generation
Output Modes
- Reference: markdown files,
index.md, andsources.json. - RAG: markdown files plus chunk files and
manifest.json. - Training: scraped source files plus
training-data.jsonlandtraining-metadata.json. - Docs mirror: complete markdown mirror with a table of contents.
Final Deliverable
# Knowledge Base: [Source]
## Summary
[What was collected and why]
## Output Structure
[Files/directories created]
## Coverage
[Sections, source types, counts]
## Usage Notes
[How to use in RAG, docs, training, or agent context]
## Sources
[URLs collected]
## Rerun Inputs
workflow: firecrawl-knowledge-base
source: [url/topic]
goal: [reference/rag/train/docs]
depth: [quick/thorough/exhaustive]
output_dir: [.firecrawl/]
Quality Bar
- Preserve code examples and formatting.
- Remove boilerplate navigation where possible.
- Include source URLs in frontmatter or metadata.
FAQ
What does the firecrawl-knowledge-base skill do?
Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources.
How do I install firecrawl-knowledge-base?
Run npx skills add https://github.com/firecrawl/firecrawl-workflows --skill firecrawl-knowledge-base in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/firecrawl-knowledge-base/ (personal) or .claude/skills/firecrawl-knowledge-base/ (project).
Which AI agents can use firecrawl-knowledge-base?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is firecrawl-knowledge-base safe to install?
Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk flagged items to review. As with any skill, read the SKILL.md and bundled scripts before installing.
Who maintains firecrawl-knowledge-base?
It is published by firecrawl in firecrawl/firecrawl-workflows, and it is an official skill from the maker of the technology it covers. It has 32,585 recorded installs.
More from firecrawl/firecrawl-workflows
firecrawl-deep-research
firecrawl/firecrawl-workflows
Produce an intensive, cited analytical report: executive summary, multi-angle findings, contrarian views, open questions, and full sources. Use only when the user needs rigorous synthesis of a complex topic (scientific, technical, policy, or market-analytical) that cannot be answered with a short search, and wants a formal written report, not a recommendation list. Do not use for product picks, top-N lists, quick lookups, or routine "find out about X" tasks. If the request does not clearly need this kind of report, do not use this skill. Do not use for a literature review over published papers. This skill collects evidence from the open web. A request for the literature on a biomedical, clinical, life-science, or other scientific topic — papers, studies, trials, preprints — belongs to firecrawl-research-papers, which queries Firecrawl's paper index (PubMed, bioRxiv, medRxiv, arXiv) instead of searching websites.
firecrawl-research-papers
firecrawl/firecrawl-workflows
Find and synthesize research papers, whitepapers, PDFs, technical reports, and academic sources with Firecrawl Research, using semantic paper search, related-paper expansion, and in-body verification over Firecrawl's paper index — largely biomedical and life-science literature from PubMed, bioRxiv, and medRxiv, plus arXiv preprints in CS, physics, and math. Use when the user wants a literature review, systematic review, survey of studies, paper summary, research landscape, or sourced synthesis from scholarly and industry publications, including clinical, drug, gene, disease, epidemiology, and public-health topics. Prefer this over a general web-research workflow whenever the evidence base is published papers rather than web pages.
firecrawl-website-design-clone
firecrawl/firecrawl-workflows
Extract an existing website's visual system into DESIGN.md, or collect its existing logo, colors, fonts and homepage messaging for a document. Use for reference-based UI implementation, documenting design patterns, or matching a presentation to a company's website. Excludes inventing a new brand or brand positioning.
firecrawl-workflows
firecrawl/firecrawl-workflows
Run focused Firecrawl workflows for research, documentation lookup, SEO audits, QA, lead discovery and enrichment, business contacts, current jobs, fantasy decisions, brand assets, knowledge bases and website design systems. Use when the user wants a sourced answer or deliverable from a workflow rather than raw extraction or product-code integration.
firecrawl-market-research
firecrawl/firecrawl-workflows
Research markets and companies with Firecrawl using current, cited figures: market size, growth, trends, financials and earnings. Use when the user asks for market research, an industry overview or a comparison of companies by the numbers, since these figures change faster than training data.
firecrawl-seo-audit
firecrawl/firecrawl-workflows
Inspect a website's titles, meta descriptions, headings, canonicals, structured data and internal links for on-page and technical SEO issues. Use for SEO audits, page-source checks, broken-link reviews or comparing pages for target keywords, even without an explicit request to scrape. Return source-backed findings and prioritized fixes. Excludes generic SEO explanations, standalone copywriting and functional website testing.
Popular docs & writing skills
grill-with-docs
mattpocock/skills
A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go.
lark-markdown
open.feishu.cn
飞书 Markdown：查看、创建、上传、编辑和比较飞书中的原生 Markdown 文件。当用户要操作飞书 Markdown 文件，或比较其远端版本及本地草稿时使用。纯本地 Markdown 文件操作不触发本 skill。不负责将 Markdown 导入为飞书在线文档，也不负责文件搜索、权限、评论、移动、删除等云空间管理操作。
writing-beats
mattpocock/skills
Writing, exploit; assemble raw material into a journey of beats, grounding each term before a beat leans on it.
writing-shape
mattpocock/skills
Writing, exploit: shape raw material into an article, paragraph by paragraph.
writing-fragments
mattpocock/skills
Writing, explore: mine raw fragments, no structure yet.
writing-for-agents
mattpocock/skills
Writing documents for agents. Use when creating or editing skills, or modifying AGENTS.md or CLAUDE.md.