sensitivity-analysis
Conduct sensitivity analyses to test robustness of findings. Use when: (1) Testing assumption violations, (2) Meta-analysis robustness, (3) Handling missing data, (4) Examining outliers.
When & Why to Use This Skill
The Sensitivity Analysis skill is designed to rigorously evaluate the robustness and reliability of research findings. By systematically testing how results fluctuate under different analytical decisions, statistical assumptions, and data conditions, this tool helps researchers and data scientists identify potential biases and ensure that their conclusions are not merely artifacts of specific methodological choices.
Use Cases
- Outlier Impact Assessment: Determine if specific extreme values are disproportionately influencing your results by performing exclusion analyses and comparing 'with' and 'without' scenarios.
- Statistical Model Validation: Test the stability of your findings by applying alternative statistical methods, such as comparing parametric tests against non-parametric alternatives or using different data transformations.
- Missing Data Robustness: Evaluate the integrity of your conclusions by simulating various missing data treatments, including best-case/worst-case scenarios and multiple imputation techniques.
- Meta-Analysis Reliability: Conduct 'one-study-removed' analyses in systematic reviews to ensure that the overall effect size is consistent and not driven by a single high-risk-of-bias study.
- Measurement Sensitivity: Verify if results remain significant when using alternative outcome definitions, different measurement time points, or varying scoring methodologies.
| name | sensitivity-analysis |
|---|---|
| description | "Conduct sensitivity analyses to test robustness of findings. Use when: (1) Testing assumption violations, (2) Meta-analysis robustness, (3) Handling missing data, (4) Examining outliers." |
| allowed-tools | Read, Write, Bash |
| version | 1.0.0 |
Sensitivity Analysis Skill
Purpose
Test whether findings are robust to analytical decisions and assumptions.
Types of Sensitivity Analyses
1. Exclusion Analyses
- Remove outliers
- Remove high risk-of-bias studies
- One-study-removed analysis
2. Analytical Decisions
- Different statistical tests
- Parametric vs non-parametric
- Different transformations
3. Missing Data
- Complete case analysis
- Best-case scenario
- Worst-case scenario
- Multiple imputation
4. Measurement
- Different outcome definitions
- Different time points
- Alternative scoring methods
Interpretation
Robust Findings:
- Results consistent across analyses
- Conclusions unchanged
- High confidence
Sensitive Findings:
- Results vary by decision
- Interpret with caution
- Report uncertainty
Example
"Results were robust to removal of the highest risk-of-bias study (d=0.48 vs d=0.52) and remained significant when using non-parametric tests (p=.002)."
Version: 1.0.0