Free
A/B Test
Calculator
The A/B Test Calculator helps you plan and analyze experiments with precision. It calculates sample size, test duration, and statistical significance, ensuring your A/B tests are backed by solid data for confident decision-making.

Pre-Test Calculator – MDE
- What is the minimum detectable effect (MDE) to consider?
- How long should the test run to detect meaningful results?
- What’s the sample size required for this test?
- How does the confidence level influence the test duration?
How to Use?
This section helps you estimate the required sample size for a valid experiment and the time needed to run your A/B test.
- Input your data:
- Confidence Level: Keep it at 95% for industry-standard confidence.
- Statistical Power: 80% is commonly used, but you can adjust.
- Conversion rate for control: Enter the current conversion rate of your control group (e.g., 10%).
- Number of variants: Specify how many test variants you are comparing against the control.
- Weekly Conversions: The number of conversions you typically get in a week. Used to get an estimate of how many weeks you need to run your experiment.
- Calculate your MDE:
Click Calculate to find out:- Minimum Detectable Effect (MDE): The smallest effect size you aim to detect in a study.
- Significance: Represents a meaningful difference you don’t want to miss if it exists.
- Expression: Entered as a percentage, either relative or absolute.
- Relation to Control: MDE is either relative to or an absolute difference from the control conversion rate.
- Example:
- Control conversion rate = 10%
- Desired minimum test variant conversion rate = 15%
- MDE absolute = 5% – (15 – 10)
- MDE relative = 50% – (15 – 10) / 10 × 100
Data Input
Results
| Sample Size per Group: | |
| Total Sample Size: | |
| Estimated Duration (weeks): |
Test Result Calculator
Once you’ve collected all your data, you can test if your variant is significantly different from the control. Before using this calculator, ensure the following:
- Required Sample Size: The test has reached the necessary sample size.
- Avoid Multiple Testing: Test only once you have collected all your data. Testing multiple times increases the risk of Type I errors by inflating the chance of finding a significant result by chance. Stick to your planned sample size and test duration before analyzing.
How to Use?
This section helps you analyze the performance of your A/B test to determine if your results are statistically significant.
- Input the following data:
- Control Visitors: Enter the total number of visitors who saw the control version of the experiment.
- Control Conversions: Enter the number of conversions (e.g., purchases, sign-ups) from those who saw the control version.
- Variant Visitors: Enter the total number of visitors who saw the test variant.
- Variant Conversions: Enter the number of conversions from those who saw the test variant.
- Confidence Level (%): Keep it at 95% (industry standard) or adjust if needed.
- Click “Calculate” to view the results:
The results will display important metrics, including the conversion rates, lift, confidence interval of the difference between, p-value, z-score, and whether the results are statistically significant. We suggest looking at the 1-sided tests since they are more relevant for A/B testing.
Data Input
Results
| Control Conversion Rate | – |
| Variant Conversion Rate | – |
| Lift (%) | – |
| Absolute differences | |
| Absolute difference | – |
| Confidence Interval (Difference, %) | – |
| Right-Sided Interval (%) | – |
| Left-Sided Interval (%) | – |
| Value ± SE (%) | – |
| P-Value (One-sided) | – |
| P-value (Two-sided) | – |
| Z-Score | – |
| Significance (One-sided) | – |
| Bayesian Probability: Variant Wins (%) | – |
| Bayesian Probability: Control Wins (%) | – |
| Bayes Factor (H1/H0) | – |
| Relative differences | |
| Relative Confidence Interval (Difference, %) | – |
| Relative Right-Sided Interval (%) | – |
| Relative Left-Sided Interval (%) | – |
| Relative Difference ± SE | – |
| Relative P-Value | – |
| Relative Z-Score | – |
A/B test calculator FAQ
How do I calculate the sample size for an A/B test?
Use the Pre-Test Calculator above. Enter your confidence level (usually 95%), statistical power (usually 80%), your control conversion rate and the smallest lift you want to be able to detect (the MDE). The calculator returns the sample size per group and in total. The smaller the effect you want to detect, the more visitors you need.
How long should an A/B test run?
Until it reaches the sample size you calculated before the test, and for at least one or two full weeks so every weekday is represented. Enter your weekly traffic in the Pre-Test Calculator to get an estimated duration in weeks. Don’t stop a test early because it looks like a winner.
What is the minimum detectable effect (MDE)?
The MDE is the smallest difference between control and variant that your test is set up to detect reliably. It can be relative (a 10% lift on a 5% conversion rate means 5.5%) or absolute (5% to 6% is a 1 percentage point lift). Low-traffic sites usually need a large MDE, which means only big changes are worth testing.
How do I know if my A/B test result is statistically significant?
Once the test has reached its planned sample size, enter visitors and conversions for both versions in the Test Result Calculator. It returns the p-value, the confidence interval of the difference and whether the result is significant at the confidence level you chose. A p-value below 0.05 at 95% confidence is the usual threshold.
Should I use a one-sided or two-sided test?
A one-sided test only asks whether the variant is better than the control, which is usually the business question, and it needs fewer visitors. A two-sided test also detects a variant that is worse. Decide before the test starts and don’t switch after you see the data.
For the full process, read the A/B testing guide and our explainer on statistical power.
Become
Certified
at CRO
Become a certified expert at conversion optimization with this
comprehensive online course.
- Learn how to build and run world-class optimization programs
- Master everything there is to master about A/B testing
- Immediately apply learnings with hundreds of playbooks
- Drive better results without increasing costs


Marketers choose CXL

The newsletter for the marketing engineer.
Free, weekly. The AI tactics, B2B marketing playbooks, and workflows that are working right now.
Join 100,000+ marketers. Unsubscribe anytime.