fbpx

A/B Testing Statistics Course

Get accurate and reliable results with statistics for A/B testing

3h 30min | 15 lessons | 16 downloadable resources

In this course, you will learn how to:

  • Apply statistical methods to ensure your A/B test results are reliable.
  • Avoid common mistakes that lead to misleading or inconclusive test data.
  • Determine the right sample size and test duration for accurate results.
  • Interpret statistical significance and confidence levels correctly.
  • Use data to make informed business decisions, not just guesses.

Black Friday offer: 25% off, already applied. A year of CXL, this course included.

English subtitles Certificate included
CXL-Rating-Transparent
Trusted by marketers at
GoogleShopifyAdobeCiscoHubSpotSpotifyAmazonMicrosoft

Master statistics for A/B testing to ensure reliable and actionable test results

Course Length:
3h 30min. 15 lessons.

Data-driven decision-making starts with understanding the numbers. This course simplifies the key statistical concepts behind experimentation, so you can confidently interpret results and avoid costly testing mistakes. Walk away knowing how to run tests that lead to improvements, not just random fluctuations. Learn to calculate sample sizes, understand statistical significance, and address complex testing scenarios to make better decisions and improve outcomes.

After completing this online course on statistics for A/B testing you will be able to assist your team and company with:

  • Designing A/B tests that provide statistically valid results.
  • Identifying false positives, errors, and misleading data before acting on it.
  • Determining when a test has reached significance and when to keep running it.
  • Calculating the right sample size for reliable decision-making.
  • Using statistics to improve the accuracy and impact of experimentation.

This course is for you if you are responsible for:

  • Running and analyzing A/B tests to improve business outcomes.
  • Making data-driven decisions that impact marketing, UX, or product strategies.
  • Ensuring test results are valid and not just statistical noise.
  • Avoiding costly mistakes caused by misinterpreting experiment data.
  • Improving your ability to communicate testing insights with confidence.

Look no further than CXL, this is the best and most recommended course you will find on Statistics for A/B Testing.

The course was updated in 2025 with actionable lessons about analyzing and presenting data and interpreting the data correctly:

Subscribe now and get access

Join for $1,199

Black Friday offer: 25% off, already applied. A year of CXL, 80+ courses included.

Course Curriculum

Statistics for A/B testing

Basics of Causal Inference

Understanding data can be challenging because real-world information is often messy and unclear.

Statistical Significance & Other Estimates

Statistical results can vary by chance, so it is important to measure how confident we are in our conclusions.

Statistical Power & Sample Size Calculations

Running an A/B test requires enough data to detect meaningful differences.

Multivariate Tests

Testing multiple variations at once can provide more insights but also brings new challenges.

Running Concurrent A/B Tests

Running multiple A/B tests at the same time can introduce challenges that affect results and common solutions often fall short.

Tests With Multiple Outcomes

Explore test setups with multiple outcomes.

Non-binomial data

Not all A/B testing data follows a simple yes-or-no (binomial) structure.

Statistics for Percentage Change

In this lesson, you will learn the difference between absolute and percentage change.

Asking the Right Questions

Learn when to use a one-sided test for your experiments.

Communicating Statistical Results

Discover how to prepare stakeholders to use statistical insights.

Planning an A/B test

Set up an A/B test using what you have learned about experiment design.

Planning an A/B/n test

Apply your skills to plan an A/B/N test, allowing you to compare more than two variations.

Planning an A/B test with non-binomial data

Learn how to implement A/B tests with more complex data that is not simply ‘yes’ or ‘no’.

Analyzing and presenting test outcomes

Analyze your test results, interpret what they mean, and present your findings in a clear way.

Georgi Georgiev

Owner @ Web Focus

Georgi is an expert internet marketer and statistician working passionately in the areas of SEO, SEM and Web Analytics since 2004. He is the founder of Analytics-Toolkit.com and owner of an online marketing agency & consulting company: Web Focus LLC and also a Google Certified Trainer in AdWords & Analytics.

His special interest lies in data-driven approaches to testing and optimization in e-commerce and internet advertising and Georgi is also the author of three papers, multiple articles on A/B testing for conversion rate optimization, as well as the book “Statistical Methods in Online A/B Testing”.

This online course
comes with a
Certificate

Completing this course earns you an official CXL Statistics for A/B Testing Course Certificate to prove your achievement. CXL is recognized as an industry-leading qualification by employers worldwide, and it is an official qualification on LinkedIn that can be added to your profile.

This is what marketers said after our live workshops and courses

Derek Miller
Built on a live GTM campaign at work

“I completed my campaign engine setup for a small c real-world GTM campaign that I was working on for my job this week and presented it to my boss today… and he loved it!”

Campaign engine

Victor’s campaign engine demo
Victor
B2B eCommerce platform

Sales signed off the target in writing before the brief passed Gate 1. Five review rounds took the drafts to 87% and 96%.

Campaign engine

Alejandro Serrano
Alejandro Serrano
Co-Founder, Feral MKT

“That alone used to take me a whole day working in AI-assisted mode (more days without AI), since I had to write all the copy from scratch because I didn’t like what Claude used to give me. This time it took me about an hour.”

Campaign engine

Ken Fleischer
Ken Fleischer
Fractional CMO & GTM Strategist

“I fixed the site and Crunchbase last week and reran Build 1 this morning with the same questions, tools and judge. Accuracy went from 53% to 71%, and wrong answers went from three to none.”

AI visibility tracker

Manuela’s Campaign Engine demo
Manuela
Social club for people 60+, pre-launch

A paid first dinner, to test whether older adults will pay for it themselves. All four channels reached ship after 9 fixes.

Campaign engine

Shannon Annarella
Shannon Annarella
Senior copywriter, elective healthcare agency

“I was really impressed with how it handled the copy, including adding a medical compliance check due to the industry I work in … I loved all of the quality checks and scoring, since I usually do that type of QA work manually.”

Campaign engine

Tegan Heneke
Freelance Marketing Executive

“Currently I’m pulling reports from GA4, SourceWhale and Linkedin manually and giving it to Claude to create a dashboard for me so this (if I can get LinkedIn connected) will change that (Yay!)”

Reporting agent

Ilaria’s Campaign Engine demo
Ilaria
Clinic management software

A campaign to win 50 demo or trial requests in 7 weeks: five channels, all drafted and through Gate 2.

Campaign engine

Natalia
Natalia
E-commerce & Brand Strategist

“I built a visual plugin on top of your Marketing Brain, two actually: Visual Brain for the brand’s visual direction and Product Visual for product image storyboards … I put my own brand through it and it already gives me something I can work with.”

Own plugins on Marketing Brain

Kerry McDonough
Kerry McDonough
AI-Forward Marketing Executive

“I was in your workshop 12 hours ago. I already have a workflow live… I feel like a freaking WIZARD!”

Last cohort

Carolin’s marketing brain demo
Carolin
Fractional CMO practice, launching

Tested a new fractional CMO offer on her own website, using public data only.

Marketing Brain

Attallah Wilson
Attallah Wilson
Marketing Communications Manager, TXI

“Connecting my systems made the biggest difference. Pulling deals straight from our HubSpot CRM, plus Semrush data on our content, gave the ICP and the messaging real evidence instead of guesses.”

Marketing Brain

Ben Shenk
Ben Shenk
Director of Messaging

“Being able to identify valuable work to be done, build a workflow/agent to do it effectively and consistently, and deploy it across my team will be a skill that sets my team and I apart within the org.”

Workflows

Murillo’s marketing brain demo
Murillo
Marketing agency for health clinics

Ran the brand book through it: every claim about the buyer came back tagged hypothesis or blank.

Marketing Brain

Ekaterina Speckbacher
Growth Marketing Manager

“I built it on my own data, so I had to hide them. It worked pretty smoothly. I’m lucky that we use HubSpot for everything, so the integration was straightforward.”

Reporting agent

Shekhar Vaidya
Head of Digital Marketing

“Fix the brand’s own pages first (one sentence on one page is wrong on every tool), then earn the unbranded recommendations.”

AI visibility tracker

Ignacio Basello
Ignacio Basello
Content Strategist

“This course got me to take writing instructions seriously and tailor them to how I work. Before, I was mostly just chatting with AI. Now I’m trying to build actual workflows.”

Workflows

Enmanuel Saavedra
Took the personal OS workshop

“Months ago, I took the workshop of the second brain (personal OS) at CXL in the past and helped to all my team to adopt it as well. It was a game changer.”

Earlier workshop

Maddy L
B2B ABM marketer, SaaS

“A challenge that we have internally: if somebody asked me for a campaign I ran three years ago … we do not have a single source of truth/location to satisfy that question … I like that this brief could help with our internal challenge”

Campaign engine

Alfonso von Ziegler
Alfonso von Ziegler
Founder, Mesh Creative Studios

“Biggest learning: the review forced me to stop claiming things I hadn’t proven yet.”

Campaign engine

Here is what other marketers said

Free, weekly, on AI and B2B marketing.
AI changes how marketing works every week. We help you keep up and grow into the marketing engineer role: practical AI tactics, B2B playbooks, and the workflows teams are running right now.

  • Hidden
  • This field is for validation purposes and should be left unchanged.

Frequently asked questions

About CXL

What is CXL?

CXL is a B2B marketing training company focused on AI and conversion optimization. Marketers learn by building real work with live guidance from practitioners.

Does CXL offer AI training for B2B marketers?

Yes. AI for B2B marketing is CXL’s main focus: building AI workflows and agents for research, content, campaigns and reporting, plus visibility in AI search.

Where can I learn conversion optimization?

CXL has trained marketers in conversion optimization and experimentation for over a decade, covering customer research, A/B testing, landing pages and testing statistics.

Plans

What is the difference between the monthly, quarterly and annual plans?

The main differences are the billing period and access to live courses:

Annual: On-demand courses plus live courses, including new live courses throughout the year. Billed annually and offers the best value. Includes a 7-day refund window.

Monthly: On-demand courses only. Billed monthly.

Quarterly: On-demand courses only. Billed every 3 months at a lower monthly rate.

Quarterly and monthly plans are not covered by the refund policy. However, you can reach out to Support if you’d like to remove the automatic renewal.

What is the difference between the individual and teams plans?

Individual plans are designed for one learner.

Team plans are available for 2 or more people and include additional tools to help manage learning across your team, including:

  • Track learning progress and engagement.
  • Invite and manage team members.
  • Create learning roadmaps for individual users.
What is included in the subscription plan?

All subscription plans include:

  • 80+ certifiable B2B marketing courses and Minidegree programs.
  • Certificates that can be added to LinkedIn.
  • Progress tracking.
Are live courses and workshops included?

Live courses and workshops are included on annual plans. Monthly and quarterly plans cover the on-demand catalog and every Minidegree.

Can I switch plans later if I want to?

Yes. You can change between Monthly, Quarterly, and Annual billing plans.

If you would like to move from an individual subscription to a team plan, please contact our Customer Success team via live chat and we’ll be happy to help.

What is the difference between the Marketing Engineer cohort and the subscription?

The Marketing Engineer cohort is 8 live weeks, starting 11 January with setup. You build a working marketing system and get certified on it, not on attendance. It comes with the annual subscription.

The subscription gives you access to our library of 80+ B2B marketing and AI courses. The annual subscription also includes every live cohort and new live workshops every month.

Does the Marketing Engineer cohort include access to the full CXL course library?

Yes. The cohort comes with the annual subscription, which includes the 80+ on-demand courses and every minidegree.

It also includes every live cohort in your year, the community and every update to the systems you build.

Billing

What are the payment options available?

Most major credit cards and Paypal are accepted. If you need an invoice to pay by wire, please message our customer success team via chat for options.

Can I pay in my own currency?

All of our products are priced in USD currency. We have established a system where you can pay in your local currency.