hypothesis-testing
Guides scientific hypothesis development and testing methodology. Use when formulating research questions, developing testable hypotheses, designing experiments, or evaluating research approaches. Triggers on phrases like "hypothesis", "test if", "experiment design", "research question", "how would I test", "is it true that".
When & Why to Use This Skill
The Hypothesis Testing skill is a comprehensive framework designed to guide users through the rigorous scientific method. It facilitates the transition from initial observations to structured research questions, helping users develop testable hypotheses, identify complex variable relationships, and select robust experimental designs. By emphasizing falsifiability and confound mitigation, it ensures high academic standards and methodological integrity in any investigative process.
Use Cases
- Academic Research & Thesis Development: Narrowing down broad observations into focused, researchable questions and formulating formal null and alternative hypotheses for peer-reviewed studies.
- Experimental Design Planning: Determining the most effective research design—such as Randomized Controlled Trials (RCT) or longitudinal studies—while identifying independent, dependent, and confounding variables.
- Product & UX Testing: Applying scientific rigor to business environments by designing experiments to test new features, ensuring that results are statistically significant and not due to external biases.
- Methodology Documentation: Generating structured research outlines and pre-registration reports that specify measurement methods, prediction magnitudes, and success criteria before data collection begins.
| name | hypothesis-testing |
|---|---|
| description | Guides scientific hypothesis development and testing methodology. Use when formulating research questions, developing testable hypotheses, designing experiments, or evaluating research approaches. Triggers on phrases like "hypothesis", "test if", "experiment design", "research question", "how would I test", "is it true that". |
Hypothesis Testing Workflow
This skill guides you through rigorous hypothesis development and testing methodology.
Phase 1: Observation and Question
Starting Point Analysis
- What observation or phenomenon prompted this inquiry?
- What patterns or anomalies are you seeing?
- What existing knowledge is relevant?
Research Question Formulation
Good research questions are:
- Focused: Specific enough to answer
- Researchable: Can be investigated empirically
- Complex: Requires analysis, not just facts
- Arguable: Has multiple possible answers
Question Types
| Type | Example | Hypothesis Style |
|---|---|---|
| Descriptive | "What is X?" | Not hypothesis-driven |
| Relational | "Is X related to Y?" | Correlation hypothesis |
| Causal | "Does X cause Y?" | Causal hypothesis |
| Comparative | "Is X different from Y?" | Difference hypothesis |
CHECKPOINT: Confirm research question with user.
Phase 2: Hypothesis Construction
Hypothesis Components
If [independent variable/condition]
Then [dependent variable/outcome]
Because [theoretical mechanism]
Null vs Alternative Hypothesis
- H₀ (Null): No effect/relationship exists
- H₁ (Alternative): Effect/relationship exists
Example:
- H₀: Training method has no effect on performance
- H₁: Training method improves performance
Hypothesis Quality Check
- Is it testable with available methods?
- Is it falsifiable (can be proven wrong)?
- Does it make specific predictions?
- Is it parsimonious (simplest explanation)?
- Is it consistent with existing knowledge?
- Does it specify the mechanism?
Phase 3: Variable Mapping
Variable Identification
| Variable | Type | Operationalization |
|---|---|---|
| [Name] | Independent (IV) | [How measured/manipulated] |
| [Name] | Dependent (DV) | [How measured] |
| [Name] | Control | [How held constant] |
| [Name] | Confound | [Potential interference] |
| [Name] | Mediator | [Explains mechanism] |
| [Name] | Moderator | [Affects strength] |
Operationalization Criteria
For each variable:
- Concrete, observable indicators
- Reliable measurement method
- Valid representation of construct
- Appropriate scale (nominal, ordinal, interval, ratio)
Phase 4: Prediction Generation
Specific Predictions
From your hypothesis, derive:
- If H₁ true: [Specific observable outcome]
- If H₀ true: [Expected null result]
- Effect direction: [Increase/decrease/differ]
- Effect magnitude: [Expected size]
Boundary Conditions
- Under what conditions should hypothesis hold?
- Where might it not apply?
- What would moderate the effect?
CHECKPOINT: Validate predictions align with user's research goals.
Phase 5: Design Selection
Experimental vs Observational
Can you manipulate the IV?
├── Yes → Experimental design
│ ├── Random assignment possible? → True experiment
│ └── No random assignment? → Quasi-experiment
└── No → Observational design
├── Over time? → Longitudinal
└── Single point? → Cross-sectional
Design Options
| Design | Strengths | Limitations |
|---|---|---|
| RCT | Causal inference | Artificial, expensive |
| Quasi-experiment | More feasible | Weaker causal claims |
| Cohort | Temporal sequence | Attrition, time |
| Case-control | Efficient for rare outcomes | Recall bias |
| Cross-sectional | Quick, inexpensive | No causation |
Control Strategies
| Threat | Control Method |
|---|---|
| Selection bias | Random assignment, matching |
| History | Control group, isolation |
| Maturation | Control group, short duration |
| Testing effects | Control group, alternate forms |
| Instrumentation | Standardization, calibration |
Phase 6: Confound Mitigation
Confound Analysis
For each potential confound:
- How could it affect the DV?
- How might it correlate with the IV?
- What's the mitigation strategy?
Mitigation Strategies
| Strategy | How It Works |
|---|---|
| Random assignment | Distributes confounds equally |
| Matching | Pairs similar participants |
| Statistical control | Adjust in analysis |
| Counterbalancing | Vary order of conditions |
| Blinding | Remove bias from knowledge |
| Standardization | Same procedures for all |
Phase 7: Falsifiability Statement
Define Falsification Criteria
Specify exactly what results would falsify H₁:
- What outcome pattern rejects the hypothesis?
- What effect size is too small to matter?
- What statistical threshold applies?
Pre-registration Elements
- Hypothesis (before seeing data)
- Analysis plan (before seeing data)
- Sample size justification
- Exclusion criteria
- Success/failure criteria
Phase 8: Documentation
Output Structure
# Hypothesis Development: [Topic]
## Research Question
[Clearly stated question]
## Hypotheses
- H₀: [Null hypothesis]
- H₁: [Alternative hypothesis]
- Mechanism: [Why we expect this]
## Variables
| Variable | Type | Operationalization |
|----------|------|-------------------|
| [Name] | [Type] | [Definition] |
## Predictions
1. If H₁: [Expected outcome]
2. If H₀: [Expected outcome]
3. Effect size: [Expected magnitude]
## Design
- Type: [Design name]
- Justification: [Why this design]
## Confounds and Controls
| Confound | Risk | Mitigation |
|----------|------|------------|
| [Name] | [Level] | [Strategy] |
## Falsification Criteria
[Specific conditions that would reject H₁]
## Feasibility Notes
- Resources needed: [List]
- Ethical considerations: [List]
- Timeline estimate: [Estimate]