mutation-testing

Validate test effectiveness with mutation testing using Stryker (TypeScript/JavaScript) and mutmut (Python). Find weak tests that pass despite code mutations. Use to improve test quality.

⭐9stars🔀0forks📁View on GitHub🕐Updated Jan 1, 2026⬇️1downloads

When & Why to Use This Skill

This Claude skill enables advanced test validation through mutation testing, utilizing industry-standard tools like Stryker (for TypeScript/JavaScript) and mutmut (for Python). It goes beyond traditional code coverage by intentionally introducing small bugs (mutants) into your source code to verify if your existing test suite can detect and 'kill' them. This process helps developers identify weak assertions and gaps in logic, ensuring that tests are truly effective at catching regressions and improving overall software reliability.

Use Cases

  • Identifying 'weak' tests that contribute to high code coverage but fail to actually validate the correctness of the logic.
  • Strengthening test suites for critical business logic where a 'survived mutant' could represent a potential production bug.
  • Optimizing test quality during refactoring by ensuring that the test suite remains robust enough to catch unintended side effects.
  • Benchmarking the effectiveness of a testing strategy by targeting a high mutation score (80%+) rather than just line coverage.
  • Detecting missing boundary condition tests by analyzing mutants that survive changes to relational or arithmetic operators.
namemutation-testing
descriptionValidate test effectiveness with mutation testing using Stryker (TypeScript/JavaScript) and mutmut (Python). Find weak tests that pass despite code mutations. Use to improve test quality.
allowed-toolsBash, Read, Edit, Write, Grep, Glob, TodoWrite

Mutation Testing

Expert knowledge for mutation testing - validating that your tests actually catch bugs by introducing deliberate code mutations.

Core Concept

  • Mutants: Small code changes introduced automatically
  • Killed: Test fails with mutation (good - test caught the bug)
  • Survived: Test passes with mutation (bad - weak test)
  • Score: Percentage of mutants killed (aim for 80%+)

TypeScript/JavaScript (Stryker)

Installation

# Using Bun
bun add -d @stryker-mutator/core @stryker-mutator/vitest-runner

# Using npm
npm install -D @stryker-mutator/core @stryker-mutator/vitest-runner

Configuration

// stryker.config.mjs
export default {
  packageManager: 'bun',
  reporters: ['html', 'clear-text', 'progress'],
  testRunner: 'vitest',
  coverageAnalysis: 'perTest',
  mutate: ['src/**/*.ts', '!src/**/*.test.ts'],
  thresholds: { high: 80, low: 60, break: 60 },
  incremental: true,
}

Running Stryker

# Run mutation testing
bunx stryker run

# Incremental mode (only changed files)
bunx stryker run --incremental

# Specific files
bunx stryker run --mutate "src/utils/**/*.ts"

# Open HTML report
open reports/mutation/html/index.html

Example: Weak Test

// Source code
function calculateDiscount(price: number, percentage: number): number {
  return price - (price * percentage / 100)
}

// ❌ WEAK: Test passes even if we mutate calculation
test('applies discount', () => {
  expect(calculateDiscount(100, 10)).toBeDefined() // Too weak!
})

// ✅ STRONG: Test catches mutation
test('applies discount correctly', () => {
  expect(calculateDiscount(100, 10)).toBe(90)
  expect(calculateDiscount(100, 20)).toBe(80)
  expect(calculateDiscount(50, 10)).toBe(45)
})

Python (mutmut)

Installation

uv add --dev mutmut

Running mutmut

# Run mutation testing
uv run mutmut run

# Show results
uv run mutmut results

# Show specific mutant
uv run mutmut show 1

# Generate HTML report
uv run mutmut html
open html/index.html

Common Mutation Types

// Arithmetic Operator
// Original: a + b → a - b, a * b, a / b

// Relational Operator
// Original: a > b → a >= b, a < b, a <= b

// Logical Operator
// Original: a && b → a || b

// Boolean Literal
// Original: true → false

Mutation Score Targets

Score Quality Action
90%+ Excellent Maintain quality
80-89% Good Small improvements
70-79% Acceptable Focus on weak areas
< 60% Poor Major improvements needed

Improving Weak Tests

Pattern: Insufficient Assertions

// Before: Mutation survives
test('calculates sum', () => {
  expect(sum([1, 2, 3])).toBeGreaterThan(0) // Weak!
})

// After: Mutation killed
test('calculates sum correctly', () => {
  expect(sum([1, 2, 3])).toBe(6)
  expect(sum([0, 0, 0])).toBe(0)
  expect(sum([])).toBe(0)
})

Pattern: Boundary Conditions

// After: Tests boundaries
test('validates age boundaries', () => {
  expect(isValidAge(18)).toBe(true)   // Min valid
  expect(isValidAge(17)).toBe(false)  // Just below
  expect(isValidAge(100)).toBe(true)  // Max valid
  expect(isValidAge(101)).toBe(false) // Just above
})

Best Practices

  • Start with core business logic modules
  • Ensure 80%+ coverage before mutation testing
  • Run incrementally (only changed files)
  • Focus on important files first
  • Don't expect 100% mutation score (equivalent mutants exist)

Workflow

# 1. Ensure good coverage first
bun test --coverage
# Target: 80%+ coverage

# 2. Run mutation testing
bunx stryker run

# 3. Check report
open reports/mutation/html/index.html

# 4. Fix survived mutants
# 5. Re-run incrementally
bunx stryker run --incremental
# or: npx stryker run --incremental

See Also

  • vitest-testing - Unit testing framework
  • test-quality-analysis - Detecting test smells