vastai-performance-tuning
Optimize Vast.ai API performance with caching, batching, and connection pooling.Use when experiencing slow API responses, implementing caching strategies,or optimizing request throughput for Vast.ai integrations.Trigger with phrases like "vastai performance", "optimize vastai","vastai latency", "vastai caching", "vastai slow", "vastai batch".
When & Why to Use This Skill
This Claude skill optimizes Vast.ai API performance by implementing advanced strategies such as response caching, request batching, and connection pooling. It is designed to help developers reduce API latency, manage high-throughput integrations, and improve the overall responsiveness of GPU-accelerated applications through production-ready TypeScript patterns.
Use Cases
- Reducing API latency for frequently accessed Vast.ai instance data using local LRU or distributed Redis caching strategies.
- Consolidating multiple API requests into single batches using DataLoader to improve throughput and minimize rate-limiting issues.
- Optimizing network efficiency by configuring persistent connection pooling and keep-alive settings for Vast.ai SDK clients.
- Implementing high-performance pagination and monitoring wrappers to track API health and operation durations in real-time.
| name | vastai-performance-tuning |
|---|---|
| description | | |
| allowed-tools | Read, Write, Edit |
| version | 1.0.0 |
| license | MIT |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
Vast.ai Performance Tuning
Overview
Optimize Vast.ai API performance with caching, batching, and connection pooling.
Prerequisites
- Vast.ai SDK installed
- Understanding of async patterns
- Redis or in-memory cache available (optional)
- Performance monitoring in place
Latency Benchmarks
| Operation | P50 | P95 | P99 |
|---|---|---|---|
| Read | 50ms | 150ms | 300ms |
| Write | 100ms | 250ms | 500ms |
| List | 75ms | 200ms | 400ms |
Caching Strategy
Response Caching
import { LRUCache } from 'lru-cache';
const cache = new LRUCache<string, any>({
max: 1000,
ttl: 60000, // 1 minute
updateAgeOnGet: true,
});
async function cachedVast.aiRequest<T>(
key: string,
fetcher: () => Promise<T>,
ttl?: number
): Promise<T> {
const cached = cache.get(key);
if (cached) return cached as T;
const result = await fetcher();
cache.set(key, result, { ttl });
return result;
}
Redis Caching (Distributed)
import Redis from 'ioredis';
const redis = new Redis(process.env.REDIS_URL);
async function cachedWithRedis<T>(
key: string,
fetcher: () => Promise<T>,
ttlSeconds = 60
): Promise<T> {
const cached = await redis.get(key);
if (cached) return JSON.parse(cached);
const result = await fetcher();
await redis.setex(key, ttlSeconds, JSON.stringify(result));
return result;
}
Request Batching
import DataLoader from 'dataloader';
const vastaiLoader = new DataLoader<string, any>(
async (ids) => {
// Batch fetch from Vast.ai
const results = await vastaiClient.batchGet(ids);
return ids.map(id => results.find(r => r.id === id) || null);
},
{
maxBatchSize: 100,
batchScheduleFn: callback => setTimeout(callback, 10),
}
);
// Usage - automatically batched
const [item1, item2, item3] = await Promise.all([
vastaiLoader.load('id-1'),
vastaiLoader.load('id-2'),
vastaiLoader.load('id-3'),
]);
Connection Optimization
import { Agent } from 'https';
// Keep-alive connection pooling
const agent = new Agent({
keepAlive: true,
maxSockets: 10,
maxFreeSockets: 5,
timeout: 30000,
});
const client = new Vast.aiClient({
apiKey: process.env.VASTAI_API_KEY!,
httpAgent: agent,
});
Pagination Optimization
async function* paginatedVast.aiList<T>(
fetcher: (cursor?: string) => Promise<{ data: T[]; nextCursor?: string }>
): AsyncGenerator<T> {
let cursor: string | undefined;
do {
const { data, nextCursor } = await fetcher(cursor);
for (const item of data) {
yield item;
}
cursor = nextCursor;
} while (cursor);
}
// Usage
for await (const item of paginatedVast.aiList(cursor =>
vastaiClient.list({ cursor, limit: 100 })
)) {
await process(item);
}
Performance Monitoring
async function measuredVast.aiCall<T>(
operation: string,
fn: () => Promise<T>
): Promise<T> {
const start = performance.now();
try {
const result = await fn();
const duration = performance.now() - start;
console.log({ operation, duration, status: 'success' });
return result;
} catch (error) {
const duration = performance.now() - start;
console.error({ operation, duration, status: 'error', error });
throw error;
}
}
Instructions
Step 1: Establish Baseline
Measure current latency for critical Vast.ai operations.
Step 2: Implement Caching
Add response caching for frequently accessed data.
Step 3: Enable Batching
Use DataLoader or similar for automatic request batching.
Step 4: Optimize Connections
Configure connection pooling with keep-alive.
Output
- Reduced API latency
- Caching layer implemented
- Request batching enabled
- Connection pooling configured
Error Handling
| Issue | Cause | Solution |
|---|---|---|
| Cache miss storm | TTL expired | Use stale-while-revalidate |
| Batch timeout | Too many items | Reduce batch size |
| Connection exhausted | No pooling | Configure max sockets |
| Memory pressure | Cache too large | Set max cache entries |
Examples
Quick Performance Wrapper
const withPerformance = <T>(name: string, fn: () => Promise<T>) =>
measuredVast.aiCall(name, () =>
cachedVast.aiRequest(`cache:${name}`, fn)
);
Resources
Next Steps
For cost optimization, see vastai-cost-tuning.