fbpx

Become great at
statistics for A/B testing

3h 30min | 11 lessons | 10 downloadable resources

In this course, you will learn how to:

  • Design A/B tests with proper statistical methods and avoid common pitfalls.
  • Calculate sample size and statistical significance for more reliable tests.
  • Interpret key statistical concepts like p-values, confidence intervals, and statistical power.
  • Navigate complex testing scenarios such as multivariate tests and concurrent tests.
  • Communicate statistical results effectively to stakeholders for better decision-making.

Sign up to All-Access Buy just this course

English subtitles Certificate included
CXL-Rating-Transparent
Some companies who train their teams at CXL

Make data-driven decisions and avoid costly mistakes with reliable A/B testing strategies

Many marketers run A/B tests but lack the statistical knowledge to trust their outcomes, which can lead to misguided decisions and wasted resources. This course teaches you how to apply the right statistical methods to ensure your tests are accurate and trustworthy. By mastering these skills, you’ll gain the confidence to make informed decisions, avoid costly errors, and maximize the impact of your testing efforts, driving better results.

Master statistics for A/B testing to ensure reliable and actionable test results

Course Length:
3h 30min. 11 lessons.

This course equips you with the statistical skills needed to design, run, and interpret A/B tests with accuracy. You’ll learn to calculate sample sizes, understand statistical significance, and navigate complex testing scenarios. By the end, you’ll confidently manage experiments that produce reliable data, leading to informed decisions and improved outcomes for your business.

After completing this online course on statistics for A/B testing you will be able to assist your team and company with:

  • Designing statistically accurate A/B tests.
  • Calculating appropriate sample sizes and significance.
  • Analyzing complex test results accurately.
  • Avoiding common pitfalls in A/B testing.
  • Communicating statistical insights to stakeholders.

This course is for you if you are responsible for:

  • Conducting A/B tests to improve business performance.
  • Analyzing and interpreting test data.
  • Driving decision-making based on data.
  • Managing experimentation across marketing or product development.
  • Presenting data-backed findings to guide business strategies.

Subscribe now and get access

See Pricing

Course Curriculum

Statistics for A/B testing

1 Basics of Causal Inference

Lesson Objectives: Understand the difficulties involved in making sense of data in a noisy worldLearn how experiments help us deal with some of those difficultiesEstablish the role of statistics in making business decisionsTranslate business questions …

2 Statistical Significance & Other Estimates

Lesson Objectives: Understand the need for the p-value and what it meansLearn how to calculate the p-value for absolute difference in proportionsIntroduce confidence intervals as an alternative presentation of the discrepancy between observations and a …

3 Statistical Power & Sample Size Calculations

Lesson Objectives: Understand type II errors, the concept of statistical power of a test, and how to account for them in planning an A/B testLearn how to determine the required sample size for a simple A/B testUnderstand the relationship between power, …

4 Multivariate Tests

Lesson Objectives: Understand what challenges are posed by A/B/n designs & the different tools for addressing themUnderstand the implications of increasing the number of variants testedExplore the trade-off between testing many variants and testing …

5 Running Concurrent A/B Tests

Lesson Objectives: Explore the complications introduced by running concurrent testsUnderstand why commonly-proposed approaches do not get the job doneLearn how and if to engage in concurrent testing Lesson 5 slide deck

6 Tests With Multiple Outcomes

Lesson Objectives: Understand the difference between primary and secondary outcomesLearn how to recognize situations which require FWER control and which do notDesign tests with multiple primary and secondary outcomes Lesson 6 slide deck

7 Non-binomial data

Lesson Objectives: Understand the three types of metrics in A/B testingCut through common misconceptions about statistics based on non-binomial dataCalculate statistical estimates for non-binomial data Lesson 7 slide deck

8 Statistics for Percentage Change

Lesson Objectives: Explore the difference between absolute and relative or percentage differenceDefine a proper statistical model for percent changeCalculate statistical estimates for working with percent change outcomes Lesson 8 slide deck

9 Asking the Right Questions

Lesson Objectives: Recognize when a one-sided test is appropriateUnderstand the need for correspondence between substantive(business) and statistical hypothesisLearn how to translate different business questions to appropriate statistical hypotheses Le …

10 Communicating Statistical Results

Lesson Objectives: Learn how to prepare stakeholders for using statistical insightsExplore best practices for presenting statistical insights through graphs, stories, tables, etc. Lesson 10 slide deck

Georgi Georgiev

Owner @ Web Focus

Georgi is an expert internet marketer and statistician working passionately in the areas of SEO, SEM and Web Analytics since 2004. He is the founder of Analytics-Toolkit.com and owner of an online marketing agency & consulting company: Web Focus LLC and also a Google Certified Trainer in AdWords & Analytics.

His special interest lies in data-driven approaches to testing and optimization in e-commerce and internet advertising and Georgi is also the author of three papers, multiple articles on A/B testing for conversion rate optimization, as well as the book “Statistical Methods in Online A/B Testing”.

This online course
comes with a
Certificate

Completing this course earns you an official CXL Statistics for A/B Testing Course Certificate to prove your achievement. CXL is recognized as an industry-leading qualification by employers worldwide, and it is an official qualification on Linkedin that can be added to your profile.

Marketers choose CXL

Subscribe to CXL
and get access

✔ 120+ Highly practical and detailed marketing courses

✔ 10x Programs to give your team the competitive edge

✔ Vetted top 1% of marketing practitioners as instructors

✔ New content monthly with on-demand access

The CXL guarantee: 7-day no questions asked money-back period

Sign up now
Team Trial

Frequently asked questions

Plans

What is the difference between the monthly, quarterly and annual plans?

We offer three types of billing lengths to choose from: Monthly, Quarterly, or Annual. The longer the billing length, the cheaper the monthly cost becomes.

Typically annual plans are 40% cheaper per year than paying month-to-month.
What is the difference between the individual and teams plans?

Personal or Individual plans are meant for one person where teams have 2 or more people.

A team account has a few extras:
– Invite and manage people learning on your team.
– Create roadmaps for each individual.
– Access team learning engagement reports.
– Assign members to your team as managers to assign content and view reports.
What is included in the all-access plan?

You’ll get:
– Access to 90+ online courses.
– Earn certificates and add them to LinkedIn.
– Thousands of playbooks and resources.
– All minidegrees
– Roadmapping and progress tracking
– Premium Support
Can I switch plans later if I want to?

Yes. You can switch between different billing lengths at any time (Ex: Switch from a Monthly Plan to Annual).

If you’d like to switch from a individual plan to a team plan (or vice versa), please reach out to customer success via live chat.

Billing

What are the payment options available?

Most major credit cards and Paypal are accepted. If you need an invoice to pay by wire, please message our customer success team via chat for options.
What is parity pricing?

All of our products are priced in USD currency. In order to make our products more affordable, we have established a system where you can pay in your local currency using a special metric that adjusts the price you pay according to the purchasing power of your country.
Is there a discount available?

One of CXL’s company policies is that we don’t discount. Our focus is on the product – providing the best, highest quality content with our courses.

The listed price on the site shows the best price – you can buy with confidence.

We offer pricing based on your local currency, you’ll see this reflected on the pricing and checkout pages.

Get scale-up growth strategies in your inbox.

An expert-led newsletter focused on helping marketing teams at growth/expansion stage companies
overcome growth challenges, crush their competition, and stand out in the market.

  • Hidden
  • This field is for validation purposes and should be left unchanged.
Subscribe

Join 140,000+ marketers | Subscribe to our educational newsletter