koan-performance
Streaming, pagination, count strategies, bulk operations
When & Why to Use This Skill
The Koan Performance skill is a specialized toolkit designed to optimize software applications for high scalability and production readiness. It provides expert patterns for efficient data handling, including streaming for large datasets, high-speed count strategies, and bulk operations. By implementing these architectural best practices, developers can significantly reduce memory consumption, eliminate N+1 query issues, and ensure sub-second response times for data-intensive operations.
Use Cases
- Scenario 1: Scaling Web APIs - Implementing efficient pagination and metadata-based 'fast count' estimates for dashboards to maintain high performance on tables with millions of records.
- Scenario 2: Memory Optimization - Refactoring data processing pipelines to use asynchronous streaming instead of loading entire datasets into memory, preventing Out-Of-Memory (OOM) errors.
- Scenario 3: High-Speed Data Migrations - Utilizing bulk save and fast removal strategies (such as TRUNCATE) to accelerate batch updates and data cleanup tasks by over 200x compared to standard loops.
- Scenario 4: Database Query Refactoring - Identifying and replacing inefficient N+1 query patterns with single-query batch retrievals to reduce database load and latency.
| name | koan-performance |
|---|---|
| description | Streaming, pagination, count strategies, bulk operations |
Koan Performance
Core Principle
Optimize for scale from day one. Use streaming for large datasets, batch operations for bulk changes, fast counts for UI, and pagination for web APIs.
Performance Patterns
Streaming (Large Datasets)
// ❌ WRONG: Load everything into memory
var allTodos = await Todo.All(); // 1 million records!
// ✅ CORRECT: Stream in batches
await foreach (var todo in Todo.AllStream(batchSize: 1000))
{
await ProcessTodo(todo);
}
Count Strategies
// Fast count (metadata estimate - 1000x+ faster)
var fast = await Todo.Count.Fast(ct); // ~5ms for 10M rows
// Exact count (guaranteed accuracy)
var exact = await Todo.Count.Exact(ct); // ~25s for 10M rows
// Optimized (framework chooses)
var optimized = await Todo.Count; // Uses Fast if available
Use Fast for: Pagination UI, dashboards, estimates Use Exact for: Critical business logic, reports, inventory
Bulk Operations
// Bulk create
var todos = Enumerable.Range(1, 1000)
.Select(i => new Todo { Title = $"Task {i}" })
.ToList();
await todos.Save(); // Single operation
// Bulk removal
await Todo.RemoveAll(RemoveStrategy.Fast); // TRUNCATE/DROP (225x faster)
Batch Retrieval
// ❌ WRONG: N queries
foreach (var id in ids)
{
var todo = await Todo.Get(id);
}
// ✅ CORRECT: 1 query
var todos = await Todo.Get(ids);
Pagination
public async Task<IActionResult> GetTodos(
int page = 1,
int pageSize = 20,
CancellationToken ct = default)
{
var result = await Todo.QueryWithCount(
t => !t.Completed,
new DataQueryOptions { OrderBy = nameof(Todo.Created), Descending = true },
ct);
Response.Headers["X-Total-Count"] = result.TotalCount.ToString();
return Ok(result.Items);
}
Performance Benchmarks
| Operation | Inefficient | Efficient | Speedup |
|---|---|---|---|
| Bulk Remove (1M) | DELETE loop ~45s | TRUNCATE ~200ms | 225x |
| Count (10M) | Full scan ~25s | Metadata ~5ms | 5000x |
| Batch Get (100) | 100 queries | 1 query | 100x |
| Stream (1M) | Load all (OOM) | Stream batches | Memory safe |
When This Skill Applies
- ✅ Performance tuning
- ✅ Large datasets
- ✅ Optimization
- ✅ Production readiness
- ✅ Memory issues
- ✅ Query optimization
Reference Documentation
- Example Code:
.claude/skills/entity-first/examples/batch-operations.cs - Guide:
docs/guides/performance.md - Sample:
samples/S14.AdapterBench/(Performance benchmarks)