mutation-testing

Test quality validation through mutation testing, assessing test suite effectiveness by introducing code mutations and measuring kill rate. Use when evaluating test quality, identifying weak tests, or proving tests actually catch bugs.

⭐23stars🔀7forks📁View on GitHub🕐Updated Dec 8, 2025

When & Why to Use This Skill

This Claude skill automates mutation testing to rigorously validate the effectiveness of your test suites. While traditional code coverage only measures which lines are executed, this skill introduces intentional logic faults (mutants) into your source code to verify if your tests can actually detect them. It provides a deep analysis of test quality through mutation scores, identifies surviving mutants that signal weak assertions, and helps developers move beyond 'shallow' coverage toward true software reliability.

Use Cases

  • Evaluating Test Suite Effectiveness: Moving beyond 100% code coverage to ensure that tests actually possess the logic required to catch real-world bugs.
  • Identifying Weak Assertions: Pinpointing specific tests that execute code but fail to validate the output correctly, allowing mutants to survive.
  • Automated Test Improvement: Using AI agents to analyze surviving mutants and automatically generate missing boundary value tests or edge case assertions.
  • Critical Path Validation: Ensuring high-risk modules like payment processing or authentication have a high 'kill rate' before major production releases.
  • Refactoring Safety: Proving that a test suite is robust enough to protect the system during large-scale architectural changes or logic refactoring.
namemutation-testing
description"Test quality validation through mutation testing, assessing test suite effectiveness by introducing code mutations and measuring kill rate. Use when evaluating test quality, identifying weak tests, or proving tests actually catch bugs."
categoryspecialized-testing
priorityhigh
tokenEstimate900
agents[qe-test-generator, qe-coverage-analyzer, qe-quality-analyzer]
implementation_statusoptimized
optimization_version1.0
last_optimized2025-12-02
dependencies[]
quick_reference_cardtrue
tags[mutation, stryker, test-quality, kill-rate, assertions, effectiveness]

Mutation Testing

When validating test quality or improving test effectiveness: 1. MUTATE code (change + to -, >= to >, remove statements) 2. RUN tests against each mutant 3. VERIFY tests catch mutations (kill mutants) 4. IDENTIFY surviving mutants (tests need improvement) 5. STRENGTHEN tests to kill surviving mutants

Quick Mutation Metrics:

  • Mutation Score = Killed / (Killed + Survived)
  • Target: > 80% mutation score
  • Surviving mutants = weak tests

Critical Success Factors:

  • High coverage ≠ good tests (100% coverage, 0% assertions)
  • Mutation testing proves tests actually catch bugs
  • Focus on critical code paths first

Quick Reference Card

When to Use

  • Evaluating test suite quality
  • Finding gaps in test assertions
  • Proving tests catch bugs
  • Before critical releases

Mutation Score Interpretation

Score Interpretation
90%+ Excellent test quality
80-90% Good, minor improvements
60-80% Needs attention
< 60% Significant gaps

Common Mutation Operators

Category Original Mutant
Arithmetic a + b a - b
Relational x >= 18 x > 18
Logical a && b a || b
Conditional if (x) if (true)
Statement return x (removed)

How Mutation Testing Works

// Original code
function isAdult(age) {
  return age >= 18; // ← Mutant: change >= to >
}

// Strong test (catches mutation)
test('18 is adult', () => {
  expect(isAdult(18)).toBe(true); // Kills mutant!
});

// Weak test (mutation survives)
test('19 is adult', () => {
  expect(isAdult(19)).toBe(true); // Doesn't catch >= vs >
});
// Surviving mutant → Test needs boundary value

Using Stryker

# Install
npm install --save-dev @stryker-mutator/core @stryker-mutator/jest-runner

# Initialize
npx stryker init

Configuration:

{
  "packageManager": "npm",
  "reporters": ["html", "clear-text", "progress"],
  "testRunner": "jest",
  "coverageAnalysis": "perTest",
  "mutate": [
    "src/**/*.ts",
    "!src/**/*.spec.ts"
  ],
  "thresholds": {
    "high": 90,
    "low": 70,
    "break": 60
  }
}

Run:

npx stryker run

Output:

Mutation Score: 87.3%
Killed: 124
Survived: 18
No Coverage: 3
Timeout: 1

Fixing Surviving Mutants

// Surviving mutant: >= changed to >
function calculateDiscount(quantity) {
  if (quantity >= 10) { // Mutant survives!
    return 0.1;
  }
  return 0;
}

// Original weak test
test('large order gets discount', () => {
  expect(calculateDiscount(15)).toBe(0.1); // Doesn't test boundary
});

// Fixed: Add boundary test
test('exactly 10 gets discount', () => {
  expect(calculateDiscount(10)).toBe(0.1); // Kills mutant!
});

test('9 does not get discount', () => {
  expect(calculateDiscount(9)).toBe(0); // Tests below boundary
});

Agent-Driven Mutation Testing

// Analyze mutation score and generate fixes
await Task("Mutation Analysis", {
  targetFile: 'src/payment.ts',
  generateMissingTests: true,
  minScore: 80
}, "qe-test-generator");

// Returns:
// {
//   mutationScore: 0.65,
//   survivedMutations: [
//     { line: 45, operator: '>=', mutant: '>', killedBy: null }
//   ],
//   generatedTests: [
//     'test for boundary at line 45'
//   ]
// }

// Coverage + mutation correlation
await Task("Coverage Quality Analysis", {
  coverageData: coverageReport,
  mutationData: mutationReport,
  identifyWeakCoverage: true
}, "qe-coverage-analyzer");

Agent Coordination Hints

Memory Namespace

aqe/mutation-testing/
├── mutation-results/*   - Stryker reports
├── surviving/*          - Surviving mutants
├── generated-tests/*    - Tests to kill mutants
└── trends/*             - Mutation score over time

Fleet Coordination

const mutationFleet = await FleetManager.coordinate({
  strategy: 'mutation-testing',
  agents: [
    'qe-test-generator',     // Generate tests for survivors
    'qe-coverage-analyzer',  // Coverage correlation
    'qe-quality-analyzer'    // Quality assessment
  ],
  topology: 'sequential'
});


Remember

High code coverage ≠ good tests. 100% coverage but weak assertions = useless. Mutation testing proves tests actually catch bugs.

Focus on critical paths first. Don't mutation test everything - prioritize payment, authentication, data integrity code.

With Agents: Agents run mutation analysis, identify surviving mutants, and generate missing test cases to kill them. Automated improvement of test quality.