Become great at statistics for A/B testing
Avoid costly testing mistakes stemming from misuse and misunderstanding of statistic
If you’re not fluent in A/B testing statistics, you won’t be able to tell whether your tests suck.
Online course
By Georgi Georgiev,
Owner @ Web Focus
Course length: 3h 30min
Some of the companies that train their teams at CXL:
A lot of your “winning” tests are probably not winners at all. Learn to call bullshit when needed, and be the person who advocates proper scientific approach in your team.
In 10 sessions, you’ll learn
- How to run A/B tests with a sound statistical design in a variety of scenarios – multiple test variants, multiple outcomes, non-binomial data, and others.
- How to align the statistical design with the questions at hand to get business insights while avoiding common mistakes.
- The logic behind statistical hypothesis testing and concepts like statistical significance, confidence intervals, statistical power, and others.
Course overview
Understand the complexities involved in planning and evaluating A/B tests
Avoid costly testing mistakes stemming from misuse and misunderstanding of statistics, and improve the ROI of all your A/B testing efforts, with Georgi Georgiev’s guidance.
Skills you will master
This course is right for you if…
- You can’t define statistical significance correctly without looking it up on Google.
- Your A/B tests produce a lot of “winners,” but your clients aren’t seeing improvements.
- You’re planning and analyzing A/B tests, but you don’t understand the statistical underpinnings of the testing process.
- You’re not confident in the outcomes of your tests and are unsure how much trust to put in them
- You have an in-house statistical tool you want to improve, or you use a third-party A/B testing software you want to understand better
This course is probably not for you if…
- You are just starting with CRO and have little to no practical experience with A/B testing.
- You don’t employ A/B tests as a primary method to evaluate CRO work.
- You are a professional statistician or experimental design specialist.
Skills you should have before taking this course:
- Some experience in conversion rate optimization.
- Basic understanding of how A/B testing works.
- Some experience with an A/B testing software.
Georgi is an expert internet marketer and statistician working passionately in the areas of SEO, SEM and Web Analytics since 2004. He is the founder of Analytics-Toolkit.com and owner of an online marketing agency & consulting company: Web Focus LLC and also a Google Certified Trainer in AdWords & Analytics.
His special interest lies in data-driven approaches to testing and optimization in e-commerce and internet advertising and Georgi is also the author of three papers, multiple articles on A/B testing for conversion rate optimization, as well as the book “Statistical Methods in Online A/B Testing”.
In just 10 sessions, you’ll be able to
- Plan A/B tests by taking into account relevant error probabilities
- Correctly interpret A/B testing statistics like p-values and confidence intervals.
- Navigate the complexities of MVT, segmentation, multiple KPIs, and concurrent tests.
- Efficiently communicate A/B test statistics.
Get a lesson preview
Your full course curriculum
Statistics for A/B testing
1 Basics of Causal Inference
2 Statistical Significance & Other Estimates
3 Statistical Power & Sample Size Calculations
4 Multivariate Tests
5 Running Concurrent A/B Tests
6 Tests With Multiple Outcomes
7 Non-binomial data
8 Statistics for Percentage Change
9 Asking the Right Questions
10 Communicating Statistical Results
Show off your hard-earned skills and strengthen your profile.
LinkedIn found that people who add credentials to their profile receive 6x more profile views than those who don’t.
Add your certificates to your profile, share your accomplishments with the world, and join our alumni network.
Subscribe to CXL and get access
- 120+ courses (full list here) on all aspects of data-driven marketing (new courses added every month)
- 10 minidegrees that give you advanced level skills
Got a team, multiple users? Get our Teams All-Access plan instead.
The CXL guarantee
- 7-day no questions asked money back period
- You can immediately apply learnings and improve your results.
Need help convincing your boss?
Click here for a pitch deck, swipe files and tips.
Subscribe to CXL
Get access to this and 100+ other training programs.
or
Learn A/B Testing Statistics: FAQ
Who is this course for?
This course is right for you if…
- You can’t define statistical significance correctly without looking it up on Google.
- Your A/B tests produce a lot of “winners,” but your clients aren’t seeing improvements.
- You’re planning and analyzing A/B tests, but you don’t understand the statistical underpinnings of the testing process.
- You’re not confident in the outcomes of your tests and are unsure how much trust to put in them
- You have an in-house statistical tool you want to improve, or you use a third-party A/B testing software you want to understand better
Some careers that will especially benefit from this course are CRO specialist, growth marketer, digital marketing manager, product manager, UX designer, data scientist, software engineer, web developer, or business analyst.
What will you learn?
After taking this course, you will:
- How to run A/B tests with a sound statistical design in a variety of scenarios – multiple test variants, multiple outcomes, non-binomial data, and others.
- How to align the statistical design with the questions at hand to get business insights while avoiding common mistakes.
- The logic behind statistical hypothesis testing and concepts like statistical significance, confidence intervals, statistical power, and others.
What does the course include?
Our A/B Testing Statistics training includes 11 lessons, 10 downloadable resources, 3h 30min of video material, several assignments, and a final exam.
Are subtitles available for the lessons?
Lessons have English subtitles and transcriptions available.
Will I receive a certificate of completion?
After taking this course, you will receive the A/B Testing Statistics Certification from CXL, with credentials that you can add to your LinkedIn profile.
What is the refund policy?
All plans come with a 7-day refund period .
How long will it take for me to finish it?
This is entirely up to you – it’s how many study hours per week you can put in. This course is 3h 30min long.