firecrawl-sdk-patterns
Apply production-ready FireCrawl SDK patterns for TypeScript and Python.Use when implementing FireCrawl integrations, refactoring SDK usage,or establishing team coding standards for FireCrawl.Trigger with phrases like "firecrawl SDK patterns", "firecrawl best practices","firecrawl code patterns", "idiomatic firecrawl".
When & Why to Use This Skill
Enhance your web scraping architecture with production-ready FireCrawl SDK patterns for TypeScript and Python. This skill provides a comprehensive framework for implementing idiomatic API integrations, featuring singleton client management, robust error handling wrappers, and automated retry logic with exponential backoff to ensure high reliability and maintainability in professional environments.
Use Cases
- Implementing a thread-safe singleton client for FireCrawl to optimize resource management and API connection efficiency in TypeScript backends.
- Building resilient data extraction pipelines by wrapping FireCrawl calls in standardized error handling and automatic retry mechanisms to handle transient network failures.
- Establishing enterprise-grade coding standards for web scraping teams using idiomatic factory patterns and Zod-based runtime response validation.
- Refactoring legacy scraping scripts into modern, type-safe architectures using Python context managers and structured logging for better observability.
| name | firecrawl-sdk-patterns |
|---|---|
| description | | |
| allowed-tools | Read, Write, Edit |
| version | 1.0.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
FireCrawl SDK Patterns
Overview
Production-ready patterns for FireCrawl SDK usage in TypeScript and Python.
Prerequisites
- Completed
firecrawl-install-authsetup - Familiarity with async/await patterns
- Understanding of error handling best practices
Instructions
Step 1: Implement Singleton Pattern (Recommended)
// src/firecrawl/client.ts
import { FireCrawlClient } from '@firecrawl/sdk';
let instance: FireCrawlClient | null = null;
export function getFireCrawlClient(): FireCrawlClient {
if (!instance) {
instance = new FireCrawlClient({
apiKey: process.env.FIRECRAWL_API_KEY!,
// Additional options
});
}
return instance;
}
Step 2: Add Error Handling Wrapper
import { FireCrawlError } from '@firecrawl/sdk';
async function safeFireCrawlCall<T>(
operation: () => Promise<T>
): Promise<{ data: T | null; error: Error | null }> {
try {
const data = await operation();
return { data, error: null };
} catch (err) {
if (err instanceof FireCrawlError) {
console.error({
code: err.code,
message: err.message,
});
}
return { data: null, error: err as Error };
}
}
Step 3: Implement Retry Logic
async function withRetry<T>(
operation: () => Promise<T>,
maxRetries = 3,
backoffMs = 1000
): Promise<T> {
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await operation();
} catch (err) {
if (attempt === maxRetries) throw err;
const delay = backoffMs * Math.pow(2, attempt - 1);
await new Promise(r => setTimeout(r, delay));
}
}
throw new Error('Unreachable');
}
Output
- Type-safe client singleton
- Robust error handling with structured logging
- Automatic retry with exponential backoff
- Runtime validation for API responses
Error Handling
| Pattern | Use Case | Benefit |
|---|---|---|
| Safe wrapper | All API calls | Prevents uncaught exceptions |
| Retry logic | Transient failures | Improves reliability |
| Type guards | Response validation | Catches API changes |
| Logging | All operations | Debugging and monitoring |
Examples
Factory Pattern (Multi-tenant)
const clients = new Map<string, FireCrawlClient>();
export function getClientForTenant(tenantId: string): FireCrawlClient {
if (!clients.has(tenantId)) {
const apiKey = getTenantApiKey(tenantId);
clients.set(tenantId, new FireCrawlClient({ apiKey }));
}
return clients.get(tenantId)!;
}
Python Context Manager
from contextlib import asynccontextmanager
from firecrawl import FireCrawlClient
@asynccontextmanager
async def get_firecrawl_client():
client = FireCrawlClient()
try:
yield client
finally:
await client.close()
Zod Validation
import { z } from 'zod';
const firecrawlResponseSchema = z.object({
id: z.string(),
status: z.enum(['active', 'inactive']),
createdAt: z.string().datetime(),
});
Resources
Next Steps
Apply patterns in firecrawl-core-workflow-a for real-world usage.