ab-testing

Use when designing experiments for subject lines, offers, cadences, or journeys.

⭐31stars🔀7forks📁View on GitHub🕐Updated Dec 23, 2025⬇️1downloads

When & Why to Use This Skill

This Claude skill provides a comprehensive framework for designing, executing, and analyzing A/B tests and marketing experiments. It empowers users to move beyond guesswork by providing structured templates for hypothesis testing, sample size calculation, and statistical analysis to optimize subject lines, offers, and customer journeys.

Use Cases

  • Email Marketing Optimization: Designing rigorous experiments for subject lines and preview text to maximize open and click-through rates.
  • Conversion Rate Optimization (CRO): Testing variations in CTA placement, creative assets, and promotional offers to improve overall conversion metrics.
  • User Journey Refinement: Optimizing automation triggers, cadences, and wait times within customer lifecycles to reduce churn and increase engagement.
  • Data-Driven Decision Making: Utilizing statistical frameworks like z-tests and Bayesian uplift to validate marketing hypotheses and document institutional learnings.
nameab-testing
descriptionUse when designing experiments for subject lines, offers, cadences, or

Experimentation & A/B Testing Skill

When to Use

  • Validating new subject lines or creative.
  • Testing segmentation hypotheses (persona vs behavior).
  • Optimizing cadence, timing, or automation triggers.

Framework

  1. Hypothesis – define expected uplift + rationale.
  2. Metric Selection – primary (open/click/conv) + guardrails (unsubs, spam).
  3. Sample Sizing – ensure stat significance (min 500 recipients per variant or use power calculator).
  4. Execution – randomize, keep variants isolated, limit simultaneous tests.
  5. Analysis – use z-test or Bayesian uplift; document learnings.

Templates

  • Experiment brief (hypothesis, segments, KPI, risk guardrails).
  • Variant table (control vs test inputs, creative asset links, owner).
  • Calculator sheet for minimum detectable effect + sample size.
  • Post-test debrief doc capturing learnings + rollout plan.

Experiment Ideas

  • Subject line vs preview text combos.
  • CTA placement (hero vs footer).
  • Personalization depth (basic vs dynamic modules).
  • Wait times between touches.

Tips

  • Run no more than two tests per journey simultaneously.
  • Recycle learnings into playbooks + automation templates.
  • Segment results by persona to catch hidden signals.