koan-performance

Streaming, pagination, count strategies, bulk operations

⭐2stars🔀1forks📁View on GitHub🕐Updated Nov 11, 2025⬇️1downloads

When & Why to Use This Skill

The Koan Performance skill is a specialized toolkit designed to optimize software applications for high scalability and production readiness. It provides expert patterns for efficient data handling, including streaming for large datasets, high-speed count strategies, and bulk operations. By implementing these architectural best practices, developers can significantly reduce memory consumption, eliminate N+1 query issues, and ensure sub-second response times for data-intensive operations.

Use Cases

  • Scenario 1: Scaling Web APIs - Implementing efficient pagination and metadata-based 'fast count' estimates for dashboards to maintain high performance on tables with millions of records.
  • Scenario 2: Memory Optimization - Refactoring data processing pipelines to use asynchronous streaming instead of loading entire datasets into memory, preventing Out-Of-Memory (OOM) errors.
  • Scenario 3: High-Speed Data Migrations - Utilizing bulk save and fast removal strategies (such as TRUNCATE) to accelerate batch updates and data cleanup tasks by over 200x compared to standard loops.
  • Scenario 4: Database Query Refactoring - Identifying and replacing inefficient N+1 query patterns with single-query batch retrievals to reduce database load and latency.
namekoan-performance
descriptionStreaming, pagination, count strategies, bulk operations

Koan Performance

Core Principle

Optimize for scale from day one. Use streaming for large datasets, batch operations for bulk changes, fast counts for UI, and pagination for web APIs.

Performance Patterns

Streaming (Large Datasets)

// ❌ WRONG: Load everything into memory
var allTodos = await Todo.All(); // 1 million records!

// ✅ CORRECT: Stream in batches
await foreach (var todo in Todo.AllStream(batchSize: 1000))
{
    await ProcessTodo(todo);
}

Count Strategies

// Fast count (metadata estimate - 1000x+ faster)
var fast = await Todo.Count.Fast(ct); // ~5ms for 10M rows

// Exact count (guaranteed accuracy)
var exact = await Todo.Count.Exact(ct); // ~25s for 10M rows

// Optimized (framework chooses)
var optimized = await Todo.Count; // Uses Fast if available

Use Fast for: Pagination UI, dashboards, estimates Use Exact for: Critical business logic, reports, inventory

Bulk Operations

// Bulk create
var todos = Enumerable.Range(1, 1000)
    .Select(i => new Todo { Title = $"Task {i}" })
    .ToList();
await todos.Save(); // Single operation

// Bulk removal
await Todo.RemoveAll(RemoveStrategy.Fast); // TRUNCATE/DROP (225x faster)

Batch Retrieval

// ❌ WRONG: N queries
foreach (var id in ids)
{
    var todo = await Todo.Get(id);
}

// ✅ CORRECT: 1 query
var todos = await Todo.Get(ids);

Pagination

public async Task<IActionResult> GetTodos(
    int page = 1,
    int pageSize = 20,
    CancellationToken ct = default)
{
    var result = await Todo.QueryWithCount(
        t => !t.Completed,
        new DataQueryOptions { OrderBy = nameof(Todo.Created), Descending = true },
        ct);

    Response.Headers["X-Total-Count"] = result.TotalCount.ToString();
    return Ok(result.Items);
}

Performance Benchmarks

Operation Inefficient Efficient Speedup
Bulk Remove (1M) DELETE loop ~45s TRUNCATE ~200ms 225x
Count (10M) Full scan ~25s Metadata ~5ms 5000x
Batch Get (100) 100 queries 1 query 100x
Stream (1M) Load all (OOM) Stream batches Memory safe

When This Skill Applies

  • ✅ Performance tuning
  • ✅ Large datasets
  • ✅ Optimization
  • ✅ Production readiness
  • ✅ Memory issues
  • ✅ Query optimization

Reference Documentation

  • Example Code: .claude/skills/entity-first/examples/batch-operations.cs
  • Guide: docs/guides/performance.md
  • Sample: samples/S14.AdapterBench/ (Performance benchmarks)