How to use it
- Enter the page's current conversion rate, for example a 3% inquiry rate.
- Enter the smallest lift worth shipping as the MDE.
- Enter daily visitors and the share in the test to get the sample size and the number of weeks.
The formula
Per-variant n = (z₁₋α/₂ · √(2p̄(1−p̄)) + z₁₋β · √(p₁(1−p₁) + p₂(1−p₂)))² ÷ (p₂ − p₁)², where p̄ is the mean of the two rates. With several variants α is divided by the number of comparisons (Bonferroni).
Checklist before you start
- Pick one primary goal, write it down before starting and do not change it midway.
- Fix the sample size and duration in advance and decide at the end; do not stop the first day it looks significant.
- Run at least one or two full weeks to cover weekdays and weekends.
- Check that the split is as planned and both versions display and submit forms correctly.
Frequently asked questions
How many visitors does an A/B test need?
It depends on the baseline conversion rate, the minimum detectable effect (MDE), confidence and power. At a 3% baseline, a 10% relative lift, 95% confidence and 80% power you need about 53,000 visitors per variant; a 20% lift needs about 14,000.
How do I choose the minimum detectable effect?
Ask how much lift would make the change worth shipping, then check what MDE your traffic can support. Smaller MDEs need far more visitors (roughly inversely proportional to the square of the MDE), so low-traffic sites should test bolder changes.
Why should a test run for whole weeks?
Weekday and weekend visitors often convert differently. Running in whole weeks gives every variant the full cycle and a steadier result, so the calculator rounds the duration up to whole weeks.
What if my traffic is too low for an A/B test?
Measure a goal closer to the conversion (an inquiry click rather than a sale), test bolder changes, focus on your highest-traffic pages or run longer. Do not lower the confidence to finish sooner and then treat the result as conclusive.
XunL Experiments: edit pages visually, assign visitors consistently, confirm accepted inquiries on the server and get reports with confidence intervals and a clear decision status.