statistics - ConversionXL Archives Institute Revenue optimization services Copytesting Blog Search Start 7-day trial for $1 Subscriptions Minidegrees Live sprints Online courses Free courses Resources

statistics

mouse trap

Even if your A/B tests are well planned and strategized, when run, they can often lead to non-significant results and erroneous interpretations. 

You’re especially prone to errors if incorrect statistical approaches are used.

In this post we’ll illustrate the 10 most important statistical traps to be aware of, and more importantly, how to avoid them.

Keep reading

How to Deal with Outliers in Your Data

One thing many people forget when dealing with data: outliers.

Even in a controlled online A/B test, your data set may be skewed by extremities. How do you deal with them? Do you trim them out, or is there another way?

Keep reading

A/B Testing Statistics: An Easy-to-Understand Guide

Testing tools are getting more sophisticated. Blogs are brimming with “inspiring” case studies. Experimentation is becoming more and more common for marketers. Statistical know-how, however, lags behind.

This post is filled with clear explanations of A/B testing statistics from top CRO experts. A/B testing statistics aren’t that complicated—but they are that essential to running tests correctly.

Here’s what we’ll cover (feel free to jump ahead):

  1. Mean, variance, and sampling;
  2. Statistical significance;
  3. P-values;
  4. Statistical power;
  5. Confidence intervals and margin of errors;
  6. Regression to the mean;
  7. Segmenting;
  8. Confounding variables and external factors.

And just in case you’re uncertain about why A/B testing statistics are so essential…

Keep reading

How to Avoid Being Deceived By Data

From the outside, it seems like data is impartial. It’s cold, objective, accurate.

In reality though it’s more complicated. In the hands of someone with an agenda, data can be weaponized to back up that viewpoint. Even in the hands of someone benevolent, data can be misinterpreted in dangerous ways.

Keep reading

How to Make More Money With Bayesian A/B Test Evaluation

The traditional (and most used) approach to analyzing A/B tests is to use a so-called t-test, which is a method used in frequentist statistics.

While this method is scientifically valid, it has a major drawback: if you only implement significant results, you will leave a lot of money on the table.

Keep reading

What’s on my mind

Hi, I'm Peep Laja—founder of CXL. I'm a former champion of optimization and experimentation turned business builder.

I do a lot of thinking, reading, and writing around business, strategy, and optimization. I send a weekly newsletter with what's on my mind on this stuff.

Subscribe

Categories