Institute Conversion optimization services Blog Search Start 7-day trial for $1 Subscriptions Minidegrees Online courses Upcoming courses Resources

a/b

Bayesian vs. Frequentist A/B Testing: What's the Difference?

There’s a philosophical statistics debate in the A/B testing world: Bayesian vs. Frequentist.

This is not a new debate. Thomas Bayes wrote “An Essay towards solving a Problem in the Doctrine of Chances” in 1763, and it’s been an academic argument ever since.

Keep reading

Client-Side Vs. Server-SideA/B Testing Tools: What's The Difference?

Marketers of all stripes are obsessed with tools.

This obsession has bred comprehensive lists of growth tools, SEO tools, and general online marketing tools. It is no different with us in conversion optimization. We nerd out on testing tools.

Though no optimization program has ever hinged on which tool you used, there are important distinctions between A/B testing tools—from the statistics they use, their price, and more.

One thing that is often either overlooked or misunderstood is the difference between client-side and server-side testing tools.

Keep reading

18 Top A/B Testing Tools Reviewed by CRO Experts

A/B testing is no longer a new field. Finding a proper A/B testing tool isn’t the problem anymore. Now, the problem is choosing the right one.

Keep reading

apples or oranges

A/B testing—for all the content out there about it, people still mess it up. From testing the wrong things to running A/B tests incorrectly, there are lots of ways to get it wrong.

Keep reading

results panel dashboard.

In my experience, I find that teams and organizations report many winning A/B tests with high uplifts, but somehow they don’t seem to bring those uplifts in reality. How come?

Keep reading

12 A/B Testing Mistakes I See All the Time

A/B testing is fun. With so many easy-to-use tools, anyone can—and should—do it. However, there’s more to it than just setting up a test. Tons of companies are wasting their time and money.

Keep reading

Predicting Winning A/B Tests Using Repeatable Patterns

If you ever ran a highly trustworthy and positive a/b test, chances are that you’ll remember it with an inclination to try it again in the future – rightfully so. Testing is hard work with many experiments failing or ending up insignificant. It’s optimal to try and exploit any existing knowledge for more successes and fewer failures. In our own practice we started doing just that.

Keep reading

Beyond "One Size Fits All" A/B Tests

If you’re invested in improving your A/B testing game, you’ve probably read dozens of articles and discussions on how to do a/b testing.

In reading advice about how long to run a test or what statistical significance threshold to use, you probably saw claims like “Always aim for XX% significance” or “Don’t stop a test until it reaches YYY conversions” – where XX% is usually a number higher than 95%, and YYY is usually a number higher than 100.

You might also have heard it’s best to come up with many variants to test against the control to improve your chance of finding the best option.

No matter what rule is offered, such advice seems to rest on the assumption that there is a one-size-fits-all solution that works in most situations.

Keep reading

How to Use A/B Testing and Personalization Best Together

If you ask most marketers, they will tell you that A/B testing and personalization are two completely different things. I respectfully disagree, and I think this disagreement is at the root of how to use them best together.

Keep reading

How to Segment A/B Test Results to Find Gold

You run an A/B test, and it’s a winner. Or maybe it’s flat (no difference in performance between variations). Does it mean that the treatments that you tested didn’t resonate with anyone? Probably not.

If you target all visitors with the A/B test, it merely reports overall results – and ignores what happens in a portion of your traffic, in segments.

Keep reading

How to change your marketing during and after the crisis

The world has already changed. Your marketing should too.

Check out our Live Panel series with experts in four different industries.

These leaders will discuss how to navigate uncertain times in:

  • Saas & subscription – with Guillaume “G” Cabane from G2 and Patrick Campbell from ProfitWell
  • eLearning – with Dan Layfield from Codeacademy and Dan McGaw from Effin Amazing
  • B2b (coming soon)
  • Ecommerce (coming soon)
Register today

Categories