A/B/C Test Sample Size Calculator
Calculate the required sample size and test duration for three variants,
with built-in correction for multiple comparisons.
Audience
Test Setup
How to use the A/B/C test sample size calculator
Testing two variations against your control at once? This calculator sizes a three-variant experiment and applies a Bonferroni correction so the extra comparisons do not inflate your false-positive rate. It computes everything in your browser.
- 1
Enter your audience and baseline. Set your audience size, its refresh cadence, and your current baseline conversion rate.
- 2
Add the lift for each variation. Enter the expected lift for variation B and variation C separately, since they may aim for different effects.
- 3
Set the traffic splits. Allocate traffic across A, B, and C. The splits must add up to 100%.
- 4
Read the per-pair and overall plan. You get the required sample size for each pairwise comparison plus the total audience and duration needed for the whole test.
Key terms explained
- Multiple comparisons problem
- Every extra variation you compare adds another chance for a false positive. Three variants means three pairwise tests, each adding risk.
- Bonferroni correction
- A conservative fix that divides your significance threshold by the number of comparisons, keeping the overall false-positive rate in check.
- Pairwise comparison
- Each head-to-head test between two variations (A vs B, A vs C, B vs C). The calculator sizes each one.
- Expected lift (per variation)
- The relative improvement you expect for B and for C versus the control. Smaller lifts need more traffic.
- Total required audience
- The audience the full three-variant test needs, derived from the most demanding pairwise comparison and your traffic splits.
- Statistical power
- The probability of detecting a real effect when one exists, typically targeted at 80%.
How the A/B/C sample size is calculated
The calculator runs a two-proportion z-test for each of the three pairwise comparisons, using a Bonferroni-corrected significance level (your alpha divided by three). It then takes the most demanding comparison and your traffic splits to derive the total required audience and duration.
The underlying sample-size math, and why a smaller detectable effect costs so much more traffic, is covered in the complete guide to A/B test sample size.
Frequently asked questions
Why does an A/B/C test need more traffic than an A/B test?
Adding a third variation creates more pairwise comparisons, and the Bonferroni correction tightens the significance threshold to control false positives. A stricter threshold requires a larger sample to reach the same confidence.
What is the Bonferroni correction?
It divides your significance level by the number of comparisons. With three pairwise comparisons and a 5% threshold, each test is evaluated at roughly 1.67%, which keeps the overall false-positive rate near 5%.
How is the total required audience determined?
The calculator sizes each pairwise comparison, finds the most demanding one, and scales it up by your traffic splits so every variation reaches the sample size it needs.
Can I use uneven traffic splits?
Yes, the splits just have to sum to 100%. Bear in mind that uneven allocation is less statistically efficient and increases the total audience required.
Should I run an A/B/C test or two separate A/B tests?
An A/B/C test compares everything against one shared control simultaneously, which can be faster overall, but it needs more total traffic. If traffic is scarce, sequential A/B tests may be more practical.