sitepanda
Scrape websites with a headless browser and extract main readable content as Markdown. Use this skill when the user asks to retrieve, analyze, or summarize content from a URL or website.
When & Why to Use This Skill
Sitepanda is a high-performance web scraping skill that enables Claude to navigate websites using a headless browser and convert complex web pages into clean, structured Markdown. It is designed to bridge the gap between static knowledge and the live web, allowing for real-time data retrieval, content analysis, and automated information extraction from any accessible URL.
Use Cases
- Content Summarization: Automatically fetch and condense long-form articles, blog posts, or news reports into concise summaries.
- Technical Documentation Retrieval: Scrape and analyze developer documentation or API references to provide instant answers to technical queries.
- Market & Competitive Research: Extract key information from competitor websites, landing pages, or product descriptions for comparative analysis.
- Deep Web Crawling: Perform recursive scraping to gather comprehensive data across multiple linked pages within a specific domain.
- Data Extraction for Analysis: Convert unstructured website content into Markdown format for easier processing, data cleaning, or knowledge base integration.
| name | sitepanda |
|---|---|
| description | > |
Sitepanda (Web Scraping Tool)
Instructions
When the user provides a URL or asks for website content, use Sitepanda to scrape the page.
By default, use the following command to scrape a single page:
sitepanda scrape --silent --limit 1
If you need to perform recursive scraping (following links), you must ask the user for confirmation before starting, as it may take a long time.
Capture the output, which is returned in Markdown format.
Read and analyze the extracted content.
Respond to the user using only the relevant information from the page.
If the content is long, summarize or extract only the necessary sections.
Examples
Example 1
User request: "Please summarize the article at https://example.com/blog/post-123"
Agent behavior:
- Use Sitepanda to scrape the page
- Read the extracted Markdown
- Summarize the main points in the response
Example 2
User request: "What does this documentation page say? https://example.com/docs"
Agent behavior:
- Fetch the page using Sitepanda
- Extract key sections
- Explain the content concisely