blockchain-rpc-provider-research
Systematic workflow for researching and validating blockchain RPC providers. Use when evaluating RPC providers for historical data collection, rate limits, archive access, compute unit costs, or timeline estimation for large-scale blockchain data backfills.
When & Why to Use This Skill
This Claude skill provides a systematic, empirically-validated workflow for researching and validating blockchain RPC providers. It helps developers and data engineers move beyond unreliable documentation by using a 5-step investigation process—including automated rate-limit testing and cost-benefit analysis—to ensure reliable large-scale blockchain data collection and accurate project timelines.
Use Cases
- Case 1: Selecting the most cost-effective and performant RPC provider (e.g., Alchemy vs. Infura) for a new dApp or indexing project.
- Case 2: Estimating the exact time and compute unit costs required to backfill millions of historical blocks for data analysis.
- Case 3: Performing empirical proof-of-concept (POC) testing to verify actual rate limits and archive node access before committing to a paid subscription.
- Case 4: Troubleshooting performance bottlenecks in existing blockchain data pipelines by comparing current provider performance against industry benchmarks.
| name | blockchain-rpc-provider-research |
|---|---|
| description | Systematic workflow for researching and validating blockchain RPC providers. Use when evaluating RPC providers for historical data collection, rate limits, archive access, compute unit costs, or timeline estimation for large-scale blockchain data backfills. |
Blockchain RPC Provider Research
Overview
This skill provides a systematic, empirically-validated workflow for researching blockchain RPC providers before committing to large-scale data collection projects. Use when selecting an RPC provider for historical blockchain data backfill, evaluating rate limits, comparing free tier options, or estimating collection timelines.
Key principle: Never trust documented rate limits—always validate empirically with POC testing.
Investigation Workflow
This skill follows a 5-step workflow. Each step builds on the previous:
- Research Official Documentation - Survey provider docs, pricing, archive access
- Calculate Theoretical Timeline - Compute expected collection time from documented limits
- Empirical Validation with POC Testing - Test actual rate limits (CRITICAL STEP)
- Create Comparison Matrix - Build side-by-side provider comparison
- Document Findings and Make Recommendation - Write comprehensive analysis report
Detailed workflow: See references/workflow-steps.md for complete step-by-step guide with code examples, questions to answer, and success criteria.
Quick start: For immediate testing, jump to Step 3 (Empirical Validation) using scripts/test_rpc_rate_limits.py.
Rate Limiting Best Practices
When implementing the selected provider, use conservative targeting (80-90% of empirically validated rate) with monitoring and fallback strategy.
Full guide: See references/rate-limiting-guide.md for detailed monitoring requirements, fallback strategies, and safety margins.
Common Pitfalls
Critical mistakes to avoid: Trusting documented burst limits (always validate empirically), testing with <50 blocks, parallel fetching on free tiers, ignoring compute unit costs, and forgetting archive access restrictions.
Full guide: See references/common-pitfalls.md for detailed anti-patterns with real-world examples (e.g., LlamaRPC 50 RPS → 1.37 RPS case).
Scripts
calculate_timeline.py- Calculate collection timeline from rate limits (RPS or compute units)test_rpc_rate_limits.py- Empirical rate limit testing template
Usage guide: See scripts/README.md for detailed usage examples, configuration options, and success criteria.
References
Workflow Documentation
references/workflow-steps.md- Complete 5-step workflow with detailed guidance for each stepreferences/rate-limiting-guide.md- Best practices for conservative rate targeting and monitoringreferences/common-pitfalls.md- Anti-patterns to avoid with real-world examplesreferences/example-workflow.md- Complete case study: 13M Ethereum blocks RPC selection
Data References
references/validated-providers.md- Alchemy vs LlamaRPC vs Infura vs QuickNode empirical comparisonreferences/rpc-comparison-template.md- Template for creating provider comparison matrices
Scripts
scripts/README.md- Complete usage guide for all scriptsscripts/calculate_timeline.py- Timeline calculator (RPS and compute unit modes)scripts/test_rpc_rate_limits.py- Empirical rate limit testing template
Example Workflow
Case study: Selecting RPC provider for 13M Ethereum blocks → Alchemy chosen at 5.79 RPS (26 days timeline, 4.2x faster than LlamaRPC).
Full walkthrough: See references/example-workflow.md for complete step-by-step case study showing research, calculation, validation, comparison, and final recommendation.
When to Use This Skill
Invoke this skill when:
- Evaluating blockchain RPC providers for a new project
- Planning historical data backfill timelines
- Comparing free tier vs paid provider options
- Investigating rate limiting issues with current provider
- Estimating collection timelines for multi-million block datasets
- Validating archive node access for historical queries
- Researching compute unit or API credit costs
- Building POC before production implementation
Related Patterns
This skill pairs well with:
blockchain-data-collection-validation- For validating the complete data pipeline after provider selection- Project scratch investigations in
scratch/ethereum-collector-poc/andscratch/rpc-provider-comparison/