When should you use bandit tests, and when is A/B/n testing best?
Though there are some strong proponents (and opponents) of bandit testing, there are certain use cases where bandit testing may be optimal. Question is, when?
A/B testing splits traffic 50/50 between a control and a variation. A/B split testing is a new term for an old technique—controlled experimentation.
Yet for all the content out there about it, people still test the wrong things and run A/B tests incorrectly.
A/B testing is no longer a new field. Finding a proper A/B testing tool isn’t the problem anymore. Now, the problem is choosing the right one.
There’s a philosophical statistics debate in the A/B testing world: Bayesian vs. Frequentist.
This is not a new debate. Thomas Bayes wrote “An Essay towards solving a Problem in the Doctrine of Chances” in 1763, and it’s been an academic argument ever since.
Chances are, you’ve heard of Google Optimize by now. It’s Google’s solution for A/B testing and personalization. Over the years, it has become a popular solution for optimizers around the world who wanted a freemium tool to do A/B testing.
In this post, you will learn what you can really expect from this tool. How do you configure it properly? How do you run your first experiment? Let’s go into details:
Color is an essential part of how we experience the world. But do colors really matter for conversion optimization? Can a button color guarantee better performance for a call to action (CTA)?
No single color is better than another. Ultimately, what matters is how much a button color contrasts with the area around it.
Lots of entrepreneurs struggle with pricing. How much to charge? It’s clear that the right price can make all the difference—too low and you miss out on profit; too high and you miss out on sales.
A/B testing is fun. With so many easy-to-use tools, anyone can—and should—do it. However, there’s more to it than just setting up a test. Tons of companies are wasting their time and money.
If you’re doing it right, you probably have a large list of A/B testing ideas in your pipeline. Some good ones (data-backed or result of a careful analysis), some mediocre ideas, some that you don’t know how to evaluate.
We can’t test everything at once, and we all have a limited amount of traffic.
You should have a way to prioritize all these ideas in a way that gets you to test the highest potential ideas first. And the stupid stuff should never get tested to begin with.
How do we do that?