On January 23, Google announced that, “If a web page listing is elevated into the featured snippet position, we no longer repeat it in the first page of results.”
Google Analytics helps us identify conversion uplift opportunities. Traffic is precious, and we don’t want to waste it on tests that don’t result in learning or uplifts.
However, in November 2018, Google released the functionality to all users. Since then, Google Optimize has become a primary platform to initiate personalized experiences. But GTM is still critical to overcome its enduring limitations.
How do you coordinate the analytics setup of a web shop that sells their products all over the world—if you have to handle 10+ languages and currencies in over 80 countries?
Having a well-thought-out plan for A/B testingFacebook ad campaigns is essential if you want to improve your performance reliably and consistently.
And the more you test, the better. A study of 37,259 Facebook ads found that “most companies only have one ad, but the best had hundreds.”
A/B testing Facebook ad campaigns can get complicated quickly (and easily produce invalid results). Spending the time upfront to perfect your testing process and structure will go a long way.
It’s said that, “an ounce of prevention is worth a pound of cure.” In the case of Google Ads, it’s a bit more like, “a few hours of research is worth hundreds of thousands of dollars.”
Negative keywords are easy to overlook, but they can be critical to finding your target market with PPC.
Multivariate testing (MVT) is an experiment that tests changes to several elements of a page at the same time, such as the headline, image and call to action, to find out which combination performs best and how much each element contributes. An A/B test compares whole versions of a page; a multivariate test compares every combination of the elements you change. That makes it powerful and traffic-hungry: three headlines, two images and two buttons already make 12 combinations, each of which needs its own sample.
A/B testing vs multivariate testing vs split URL testing
A/B or A/B/n test
Multivariate test
Split URL test
What changes
One or more whole versions of a page
Several elements at once, in every combination
Two different URLs, often full redesigns
Question it answers
Which version wins?
Which combination wins, and which element matters most?
Does the new page beat the old one?
Traffic needed
Sample per version x number of versions
Sample per combination x number of combinations, usually far more
Same as an A/B test
Best for
Most tests, especially on low or moderate traffic
High-traffic pages where you want to tune several elements
Redesigns and back-end changes
Main risk
Learning less about why a version won
Underpowered results and false winners across many combinations
Redirects and tracking errors
The traffic math decides most of the time. If a page needs about 2,000 visitors per variation to detect the effect you care about, an A/B test with two versions needs about 4,000 visitors. A 12-combination multivariate test needs about 24,000. Check your own numbers with the A/B test calculator before you choose. For the fundamentals, start with the A/B testing guide.
When should you use multivariate testing, and when is A/B/n testing best?
It's called marketing engineer. US companies hiring for it ask to see what you've built. In CXL's live cohort, you build a working AI marketing system on your own data, and you're certified on that system, not on attendance.
Black Friday: a year of CXL for $1,599 $1,199. Includes the January cohort and every cohort after it.
Join 140,000+ marketers and get a weekly expert-led newsletter focused on helping marketing teams overcome growth challenges, punch above their weight, and crush their competition.