A/B Testing Statistics: An Easy-to-Understand Guide
Testing tools are getting more sophisticated. Blogs are brimming with “inspiring” case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind.
This post is filled with clear explanations of A/B testing statistics from top CRO experts. A/B testing statistics aren’t that complicated—but they are that essential to running tests correctly.
Here’s what we’ll cover (feel free to jump ahead):
- Mean, variance, and sampling;
- Statistical significance;
- P-values;
- Statistical power;
- Confidence intervals and margin of errors;
- Regression to the mean;
- Segmenting;
- Confounding variables and external factors.
And just in case you’re uncertain about why A/B testing statistics are so essential…