ab-testing
A/B testing patterns for lead generation sites. Client-side experiments, Google Optimize alternative, analytics integration. Use for conversion optimization.
When & Why to Use This Skill
This Claude skill provides a robust framework for implementing A/B testing and conversion rate optimization (CRO) specifically for lead generation websites. It offers a privacy-friendly, client-side alternative to tools like Google Optimize, focusing on statistical rigor, GA4 integration, and flicker-free user experiences to ensure data-driven growth decisions.
Use Cases
- Optimizing Call-to-Action (CTA) performance by testing variations in button text, colors, and placement to maximize lead capture.
- Reducing form friction and abandonment by experimenting with different form lengths, field requirements, and layout structures.
- Improving landing page conversion rates by validating the impact of benefit-focused vs. problem-focused headlines and hero imagery.
- Implementing high-integrity experiments with built-in statistical significance checks (p < 0.05) and minimum duration safeguards to avoid false positives.
- Integrating client-side experiment data with Google Analytics 4 (GA4) using custom dimensions for detailed event tracking and conversion analysis.
| name | ab-testing |
|---|---|
| description | A/B testing patterns for lead generation sites. Client-side experiments, Google Optimize alternative, analytics integration. Use for conversion optimization. |
A/B Testing Skill
Purpose
Enables experimentation on lead generation sites to optimize conversion rates through simple, privacy-friendly client-side testing with statistical rigor.
Core Rules
- One test at a time — Per page, avoid interaction effects
- Statistical significance — Don't call winners too early (p < 0.05)
- Track conversions — Not just clicks or exposures
- Minimize flicker — Apply variants before paint using visibility hidden
- Respect privacy — No PII in experiments, localStorage only
- Minimum duration — Never less than 1 week (day-of-week effects)
- Sample size first — Calculate required n before starting
- Persist assignments — User sees same variant across sessions
- Track exposures — Log every variant view to GA4
- Conversion-focused — Optimize for business metrics, not vanity metrics
Test Duration Guidelines
| Traffic/Day | Min Duration | Sample Size |
|---|---|---|
| 100 | 8+ weeks | ~400 per variant |
| 500 | 2 weeks | ~400 per variant |
| 1000+ | 1 week | ~400 per variant |
Never call a test in less than 1 week — Day-of-week effects matter.
Common Tests for Lead Gen
| Test | Variants |
|---|---|
| CTA Text | "Get Quote" vs "Get Free Quote" vs "Start Now" |
| CTA Color | Primary vs Accent vs Contrasting |
| Form Length | 3 fields vs 5 fields |
| Social Proof | With reviews vs Without |
| Urgency | None vs "Limited slots" |
| Hero Image | Photo A vs Photo B |
| Headline | Benefit-focused vs Problem-focused |
GA4 Custom Dimensions
Set up in GA4 for tracking:
| Dimension | Scope | Description |
|---|---|---|
ab_test_id |
Event | Test identifier |
ab_variant |
Event | Variant name |
References
Detailed implementation guides and code examples:
- Variant System — ABTest and ABVariant components, usage examples
- GA4 Integration — Conversion tracking, GTM configuration, custom dimensions
- Statistics — Sample size calculator, results analysis, significance testing
Forbidden
- ❌ Calling winners without significance (p < 0.05)
- ❌ Running less than 1 week
- ❌ Multiple tests on same element
- ❌ Changing tests mid-run
- ❌ Not tracking actual conversions
- ❌ Flicker (variant change visible to user)
- ❌ Storing PII in localStorage or events
- ❌ Skipping sample size calculation
Definition of Done
- A/B test component implemented (ABTest.astro)
- Variant content component implemented (ABVariant.astro)
- Variant assignment persisted in localStorage
- Exposures tracked in GA4 (ab_test_exposure event)
- Conversions tracked in GA4 (ab_test_conversion event)
- No visible flicker (visibility: hidden until variant applied)
- Sample size calculated based on baseline conversion rate
- Test runs minimum 1 week before analysis
- GA4 custom dimensions configured (ab_test_id, ab_variant)
- Statistical significance verified (p < 0.05) before declaring winner