benchmark
Run BenchmarkDotNet performance tests and analyze results. Use when the user mentions benchmarks, performance testing, mining speed, or wants to compare performance.
When & Why to Use This Skill
This Claude skill automates performance benchmarking for .NET applications using BenchmarkDotNet. It enables developers to execute precise performance tests, analyze critical metrics such as mean execution time and variance, and perform baseline comparisons to identify regressions or validate code optimizations.
Use Cases
- Performance Regression Detection: Run benchmarks before and after code changes to ensure new features or refactors haven't negatively impacted system speed.
- Optimization Verification: Quantify the actual performance gains of algorithmic improvements, such as mining speed enhancements, using statistically significant data.
- Environment Stability Auditing: Analyze variance and standard deviation metrics to identify external performance bottlenecks like thermal throttling or background resource contention.
- Automated Reporting for PRs: Generate standardized performance reports in Markdown, CSV, or HTML formats to provide data-driven evidence for pull request reviews.
| name | benchmark |
|---|---|
| description | Run BenchmarkDotNet performance tests and analyze results. Use when the user mentions benchmarks, performance testing, mining speed, or wants to compare performance. |
| allowed-tools | Read, Bash, Glob, Grep |
Performance Benchmarking
Run and analyze mining performance benchmarks using BenchmarkDotNet.
Quick Start
Run full benchmark suite:
dotnet run -c Release --project src/ZChain.PerformanceTesting/ZChain.PerformanceTesting.csproj
Benchmark Parameters
| Parameter | Values | Description |
|---|---|---|
| ThreadCount | 1, 2, 3, 10 | Number of parallel mining threads |
| Difficulty | 1, 2, 3 | Leading zeros required in hash |
Results Location
Results are saved to: BenchmarkDotNet.Artifacts/results/
Key files:
*-report-github.md- Markdown table for GitHub/documentation*-report.csv- Raw data for analysis*-report.html- Interactive HTML report
Interpreting Results
| Metric | Meaning | Healthy Range |
|---|---|---|
| Mean | Average execution time | Varies by difficulty |
| Error | Half of 99.9% CI | <10% of Mean |
| StdDev | Standard deviation | <25% of Mean |
| Median | Middle value | Close to Mean |
Variance Guidelines
- 1-10% variance: Excellent measurement stability
- 10-25% variance: Acceptable, minor environmental noise
- >25% variance: Investigate - close background apps, check thermal throttling
Comparing Results
To compare against baseline:
- Current baseline in
BenchmarkDotNet.Artifacts/results/*-baseline.csv - Run new benchmark
- Compare Mean values - >10% regression warrants investigation
Troubleshooting
High Variance
- Close resource-intensive applications
- Disable CPU throttling/power saving
- Ensure system isn't thermally throttling
- Run multiple times to identify outliers
BenchmarkDotNet Version
Using 0.14.0 (pinned). Version 0.15.x showed unstable measurements (5-34% variance vs 1-21% with 0.14.0).
Workflow
- Before changes: Run baseline benchmark
- Make changes: Implement feature/optimization
- After changes: Run comparison benchmark
- Analyze: Compare Mean times, check variance
- Document: Note significant changes in PR description