web-research
Web research using fetch_webpage and Playwright MCP tools in headless mode
When & Why to Use This Skill
This Claude skill enables high-performance automated web research through a sophisticated two-tier approach. It prioritizes speed using direct webpage fetching for static content and seamlessly falls back to headless Playwright browser automation for complex, JavaScript-heavy sites or interactive search tasks. By enforcing strict quality standards and post-research reflection, it ensures accurate, multi-sourced, and well-documented information retrieval for any AI-driven project.
Use Cases
- Technical Documentation Analysis: Efficiently fetching and synthesizing the latest API references, best practices, and version-specific details from official developer portals like React or GitHub.
- Dynamic Web Data Extraction: Navigating and scraping information from modern, JS-rendered web applications that require browser-like interactions such as clicking, typing, or waiting for elements to load.
- Comprehensive Market & Topic Research: Performing automated searches across multiple engines (e.g., DuckDuckGo) to gather and cross-reference insights from at least three independent sources.
- Knowledge Base Augmentation: Systematically discovering and 'remembering' key findings and source URLs to build a persistent, project-wide context for long-term tasks.
| name | web-research |
|---|---|
| description | Web research using fetch_webpage and Playwright MCP tools in headless mode |
Web Research Skill
Two-tier approach: fetch_webpage first → Playwright MCP tools (headless) as fallback.
See
.gaia/skills/reflection.mdfor post-research reflection process.
Tier 1: fetch_webpage (Primary)
Use for known URLs (docs, blogs, GitHub, APIs). Fast and efficient.
fetch_webpage({
urls: ["https://react.dev/learn"],
query: "React hooks best practices",
});
Limitations: No JS rendering, no searches, no interactions.
Tier 2: Playwright MCP Tools (Fallback - Headless Mode)
Use for: web searches, dynamic content, JS-rendered pages, interactions.
IMPORTANT: Playwright runs in headless mode for research. No visible browser window.
Search Pattern:
- Navigate:
mcp_playwright-mc_browser_navigate({ url: "https://duckduckgo.com" }) - Snapshot:
mcp_playwright-mc_browser_snapshot()- get element refs - Type:
mcp_playwright-mc_browser_type({ element: "search", ref: "[ref]", text: "query" }) - Submit:
mcp_playwright-mc_browser_key({ element: "search", ref: "[ref]", key: "Enter" }) - Wait:
mcp_playwright-mc_browser_wait_for({ text: "results" }) - Extract:
mcp_playwright-mc_browser_snapshot()- parse results
Key Tools: navigate, snapshot, click, type, key, wait_for, screenshot, evaluate
Quality Standards
- ✅ Minimum 3 sources
- ✅ Prioritize official docs
- ✅ Include versions & dates
- ✅
recall()before research - ✅
remember()key findings - ✅ Reflect after completion (see
.gaia/skills/reflection.md)
Output Format
✓ Research: [Topic]
**[Answer]** (v[X], [Date])
- Key finding 1
- Key finding 2
**Source**: [URL]
**Method**: fetch_webpage / Playwright headless
Post-Research Reflection
After completing research:
remember("research", "[topic]", "[key findings + sources]", "ProjectWide")
remember("source", "[topic]", "[best URLs found]", "ProjectWide")
remember("pattern", "research_[approach]", "[what worked well]", "ProjectWide")
Critical Rules
- ✅ Try fetch_webpage FIRST
- ✅ Playwright runs headless (no GUI)
- ✅ Cite sources with URLs
- ✅ Cache findings via
remember() - ✅ Reflect on research quality/efficiency
- ❌ Never skip fetch_webpage
- ❌ Never present unverified info
- ❌ Never skip post-research reflection