firecrawl

Mandalorian007's avatarfrom Mandalorian007

Web scraping and URL discovery using Firecrawl. Use for scraping web pages to markdown and discovering URLs on websites. Keywords: scrape, crawl, firecrawl, web, markdown, url discovery, sitemap.

⭐0stars🔀0forks📁View on GitHub🕐Updated Jan 3, 2026⬇️1downloads
[Web Automation]

When & Why to Use This Skill

Firecrawl is a high-performance Claude skill designed for advanced web scraping and URL discovery. It specializes in crawling entire websites and converting complex web pages into clean, LLM-ready markdown format. By automating the extraction of structured data and sitemap mapping, it enables users to transform the web into a readable knowledge source for deep analysis and research.

Use Cases

  • Markdown Content Extraction: Automatically scrape and convert blog posts, technical documentation, or articles into clean markdown for easy integration into knowledge bases.
  • Website Mapping & URL Discovery: Efficiently explore a domain's architecture to find specific sub-pages, hidden links, or comprehensive sitemaps without manual searching.
  • Competitive Intelligence: Systematically crawl competitor websites to monitor product updates, pricing changes, or public announcements for market research.
  • Data Sourcing for LLMs: Provide Claude with high-quality, noise-free web data by stripping away HTML clutter, ensuring better context management and more accurate AI responses.
nameFirecrawl
descriptionWeb scraping and URL discovery using Firecrawl. Use for scraping web pages to markdown and discovering URLs on websites. Keywords: scrape, crawl, firecrawl, web, markdown, url discovery, sitemap.

Firecrawl

Web scraping and URL discovery powered by Firecrawl API.

Variables

  • FIRECRAWL_CLI_PATH: .claude/skills/firecrawl/firecrawl_cli/

Instructions

Run from FIRECRAWL_CLI_PATH:

cd .claude/skills/firecrawl/firecrawl_cli/
uv run fc --help                   # Discover all commands
uv run fc <command> --help         # Detailed usage

Rules:

  • ALWAYS use Task agent for scrapes - pages are large and will overload context
  • Use map to explore - when you need to find where content lives on a site
  • Use Perplexity for search - this skill is for targeted scraping, not search

Troubleshooting

  • "FIRECRAWL_API_KEY not found": Run /prime to validate environment