property-based-testing

Smaug123's avatarfrom Smaug123

How to make a good property-based test. Use when writing any property-based test.

⭐0stars🔀0forks📁View on GitHub🕐Updated Dec 14, 2025⬇️3downloads
[Testing and QA]

When & Why to Use This Skill

This Claude skill provides expert guidance and best practices for implementing high-quality property-based testing. It focuses on optimizing test case generation, instrumenting input distributions to ensure edge-case coverage, and utilizing advanced fuzzing techniques to enhance software reliability and bug detection.

Use Cases

  • Efficiently generating complex data structures (e.g., non-empty lists) directly rather than filtering random data, ensuring faster and more relevant test execution.
  • Adding instrumentation to test suites to monitor and assert that the input distribution actually hits specific logic paths, preventing 'silent' test gaps where edge cases are never explored.
  • Fuzzing the parameters of data generators themselves to explore diverse data regimes and uncover bugs that only appear under specific distribution biases.
  • Improving debugging workflows by ensuring property-based tests throw descriptive exceptions on failure rather than returning simple boolean values, making failures more interpretable.
nameproperty-based-testing
descriptionHow to make a good property-based test.

Property-based testing

  • Generate correct test cases, don't filter down to correct test cases. Example: don't generate all lists and filter to non-empty ones; instead, generate an element and a list and concatenate them.
  • When writing a test that has nontrivial requirements on the input distribution, add instrumentation to it and assert the actual observed distribution. Example: if the last line of the test is someSet.Contains elt |> shouldEqual (referenceImplementation.Contains elt), we want to make sure we do hit the positive case sometimes; so additionally increment mutable variables positiveCount and negativeCount, and after the call to the testing framework to assert the property, check that the counts are in a reasonable ratio to each other. Assert this, rather than merely printing it; it's a bug in the test if the test isn't exploring the space.
  • Fuzz over the distribution itself where appropriate. You can cheaply explore very different regimes by biasing the generator according to some parameters and fuzzing over the bias parameters themselves.
  • The tests give more interpretable output if the property throws on failure, rather than merely failing an equality check. In FsCheck, use b |> shouldEqual a (a unit-returning property) rather than a = b (a bool-returning one).