fbpx

A/B Testing

The Experiment Canvas: A Better Way to Plan Tests

I have been part of some the best conversion optimization teams in the world, and they seem to have an intuitive sense on how to run the best experiments. People that are involved in these teams share a similar mindset.

I wanted to try to make this a process, one that could teach any organization how to run better experiments. I wanted to try to make this mindset more explicit in a way that is fun to use.

There’s no rocket science that follows, but this framework may well help your team drive a more efficient optimization culture.

Keep reading

Why “Only One Change Per Test” Isn’t Good Advice

Do a quick Google search for “A/B testing mistakes,” and you’ll find a good amount of articles.

Common on those lists is the oft-repeated advice that you should “not make more than one change per test.”

As it turns out, like much of the advice in the conversion optimization world, it’s not so simple.

Keep reading

User State Models

William A. Foster once said, “Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction, and skillful execution; it represents the wise choice of many alternatives.”

Yet, we continue to see businesses pushing leads through doors, pushing customers through funnels… just hoping that they’ll create a high quality, engaged audience by accident.

Unfortunately, it doesn’t work that way. A high quality, engaged audience is anything but accidental. It requires that optimizers put in the effort to create user state models, dig into cohort analysis and correlative metrics, run experiments for different user states, etc.

It’s not the easy choice, but if you’re looking for long-term revenue growth, it’s the only choice.

Keep reading

Innovative Testing

Most people stick to one end of the testing spectrum, which is where iterative testing falls. Why? Likely because it’s the most talked about, the most publicized.

However, there is another end of the spectrum to explore, which is where innovative testing falls. Understanding one without the other is a disservice to yourself, your colleagues and your boss / clients.

A solid testing program includes the full spectrum.

Keep reading

High Velocity

Mark Zuckerberg famously said, “Move fast and break things. Unless you are breaking stuff, you are not moving fast enough.”

Since then, startups and growth marketers have latched onto the statement. “Move fast and break things” has become a way of life, an ideal for aspiring entrepreneurs who just want to hustle all day, hustle all night like Gary Vaynerchuk.

But how true is that statement, which Mark made many, many years ago?

Does it apply to testing and experimentation? The philosophy of high velocity testing, made popular by a number of different testing and growth experts, certainly makes the case that it does.

Keep reading

Categories