web-scraping

tatat's avatarfrom tatat

Extract structured data from web pages using CSS selectors and XPath

⭐1stars🔀0forks📁View on GitHub🕐Updated Dec 23, 2025

When & Why to Use This Skill

This Claude skill enables automated web scraping to extract structured data from any website using CSS selectors and XPath. It simplifies complex data collection by converting raw HTML into organized JSON or CSV formats, supporting advanced features like pagination and regex pattern matching for efficient information gathering.

Use Cases

  • E-commerce Monitoring: Automatically track competitor product prices, descriptions, and stock status to inform dynamic pricing strategies.
  • Market Research: Aggregate news headlines, blog posts, or social proof from various websites to identify emerging industry trends and consumer sentiment.
  • Lead Generation: Systematically extract business contact information and company profiles from professional directories or industry-specific listings.
  • Content Aggregation: Collect and format data from multiple web sources to build comprehensive databases for research or internal reporting.
nameweb-scraping
descriptionExtract structured data from web pages using CSS selectors and XPath

Web Scraping

Extract structured data from web pages.

Capabilities

  • Fetch HTML content from URLs
  • Parse and extract specific elements (tables, lists, text)
  • Handle pagination
  • Output in JSON or CSV format

Supported Selectors

  • CSS selectors: .class, #id, tag
  • XPath expressions
  • Text patterns (regex)

Rate Limiting

Always respect robots.txt and implement delays between requests. Default delay: 1 second between requests.

Example

Scrape product names and prices from example.com/products
Output as JSON with fields: name, price, url