firecrawl-knowledge-ingest
Export documentation portals, help centers and API reference sites into local markdown or JSON with Firecrawl browser, including pages that render with JavaScript, hide content behind collapsible navigation or paginate. Use when the user wants a docs site, help center or support knowledge base saved as files, even without an explicit request to scrape.
- Installs
- 31.4K
- Last 24h
- +21
- Trending rank
- #5661
- Security
- Warn
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/firecrawl/firecrawl-workflows --skill firecrawl-knowledge-ingest- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads firecrawl-knowledge-ingest automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
Security audits
Independent scans of the skill's instructions and bundled files.
SKILL.md
The instructions your agent reads when the skill is triggered.
Firecrawl Knowledge Ingest
Use this when a docs portal needs browser navigation, auth, pagination, or JS rendering.
Onboarding Interview
Infer the portal URL, output format, auth needs, and page limit from context. If the portal is clear, proceed immediately.
Ask at most 1-3 concise questions only if blocked, such as the portal URL, whether authentication is required, or the desired output format.
Firecrawl Collection Plan
Use Firecrawl browser to:
- open the portal and inspect navigation
- identify sections, categories, sidebar links, and article URLs
- follow sidebar navigation, next links, pagination, load-more controls, or search
- scrape article content as markdown
- extract metadata such as title, section, last updated date, author, and tags
Try Firecrawl map as a supplement for public URLs, but use browser navigation for auth-gated or JS-heavy content.
Final Deliverable
# Knowledge Ingest: [Portal]
## Summary
[Pages extracted, sections covered, limitations]
## Output
[JSON/markdown/merged file path or content]
## Sections
[Section names and article counts]
## Failed Or Restricted Pages
[Any access/loading issues]
## Sources
[URLs extracted]
## Rerun Inputs
workflow: firecrawl-knowledge-ingest
url: [portal url]
format: [json/markdown/merged]
max_pages: [number]
JSON Shape
Use source, url, extractedAt, totalArticles, and sections[] with article title, url, section, content, and metadata.
Quality Bar
- Preserve code examples, tables, and formatting.
- Strip nav chrome, headers, and footers.
- Track extraction progress and page failures.
- Respect authentication boundaries.
FAQ
What does the firecrawl-knowledge-ingest skill do?
Export documentation portals, help centers and API reference sites into local markdown or JSON with Firecrawl browser, including pages that render with JavaScript, hide content behind collapsible navigation or paginate. Use when the user wants a docs site, help center or support knowledge base saved as files, even without an explicit request to scrape.
How do I install firecrawl-knowledge-ingest?
Run npx skills add https://github.com/firecrawl/firecrawl-workflows --skill firecrawl-knowledge-ingest in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/firecrawl-knowledge-ingest/ (personal) or .claude/skills/firecrawl-knowledge-ingest/ (project).
Which AI agents can use firecrawl-knowledge-ingest?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is firecrawl-knowledge-ingest safe to install?
Security partners have reviewed it: Gen Agent Trust Hub passed; Socket passed; Snyk flagged items to review. As with any skill, read the SKILL.md and bundled scripts before installing.
Who maintains firecrawl-knowledge-ingest?
It is published by firecrawl in firecrawl/firecrawl-workflows, and it is an official skill from the maker of the technology it covers. It has 31,381 recorded installs.
More from firecrawl/firecrawl-workflows
firecrawl-deep-research
firecrawl/firecrawl-workflows
Produce an intensive, cited analytical report: executive summary, multi-angle findings, contrarian views, open questions, and full sources. Use only when the user needs rigorous synthesis of a complex topic (scientific, technical, policy, or market-analytical) that cannot be answered with a short search, and wants a formal written report, not a recommendation list. Do not use for product picks, top-N lists, quick lookups, or routine "find out about X" tasks. If the request does not clearly need this kind of report, do not use this skill. Do not use for a literature review over published papers. This skill collects evidence from the open web. A request for the literature on a biomedical, clinical, life-science, or other scientific topic — papers, studies, trials, preprints — belongs to firecrawl-research-papers, which queries Firecrawl's paper index (PubMed, bioRxiv, medRxiv, arXiv) instead of searching websites.
firecrawl-research-papers
firecrawl/firecrawl-workflows
Find and synthesize research papers, whitepapers, PDFs, technical reports, and academic sources with Firecrawl Research, using semantic paper search, related-paper expansion, and in-body verification over Firecrawl's paper index — largely biomedical and life-science literature from PubMed, bioRxiv, and medRxiv, plus arXiv preprints in CS, physics, and math. Use when the user wants a literature review, systematic review, survey of studies, paper summary, research landscape, or sourced synthesis from scholarly and industry publications, including clinical, drug, gene, disease, epidemiology, and public-health topics. Prefer this over a general web-research workflow whenever the evidence base is published papers rather than web pages.
firecrawl-website-design-clone
firecrawl/firecrawl-workflows
Extract an existing website's visual system into DESIGN.md, or collect its existing logo, colors, fonts and homepage messaging for a document. Use for reference-based UI implementation, documenting design patterns, or matching a presentation to a company's website. Excludes inventing a new brand or brand positioning.
firecrawl-workflows
firecrawl/firecrawl-workflows
Run focused Firecrawl workflows for research, documentation lookup, SEO audits, QA, lead discovery and enrichment, business contacts, current jobs, fantasy decisions, brand assets, knowledge bases and website design systems. Use when the user wants a sourced answer or deliverable from a workflow rather than raw extraction or product-code integration.
firecrawl-knowledge-base
firecrawl/firecrawl-workflows
Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources.
firecrawl-market-research
firecrawl/firecrawl-workflows
Research markets and companies with Firecrawl using current, cited figures: market size, growth, trends, financials and earnings. Use when the user asks for market research, an industry overview or a comparison of companies by the numbers, since these figures change faster than training data.
Popular docs & writing skills
grill-with-docs
mattpocock/skills
A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go.
lark-markdown
open.feishu.cn
飞书 Markdown：查看、创建、上传、编辑和比较飞书中的原生 Markdown 文件。当用户要操作飞书 Markdown 文件，或比较其远端版本及本地草稿时使用。纯本地 Markdown 文件操作不触发本 skill。不负责将 Markdown 导入为飞书在线文档，也不负责文件搜索、权限、评论、移动、删除等云空间管理操作。
writing-beats
mattpocock/skills
Writing, exploit; assemble raw material into a journey of beats, grounding each term before a beat leans on it.
writing-shape
mattpocock/skills
Writing, exploit: shape raw material into an article, paragraph by paragraph.
writing-fragments
mattpocock/skills
Writing, explore: mine raw fragments, no structure yet.
writing-for-agents
mattpocock/skills
Writing documents for agents. Use when creating or editing skills, or modifying AGENTS.md or CLAUDE.md.