03 Tools03-02
A/B sample size
How many visitors each arm needs before a lift of a given size can be seen, and whether that fits in eight weeks at the traffic available.
03-02 · A/B testing · sample sizeTwo-sided · α = 0.05 · equal arms
Inputs
Limit: normal approximation for two proportions. Does not cover sequential peeking or more than two arms.
Testable. 39,473 visitors per arm reach a 10% lift in 40 days – run 6 full weeks.
Per arm
39,473
visitors
Total
78,946
two arms
Duration
40 d
6 full weeks · within the 8-week ceiling
Absolute MDE
0.40 pp
4.0% → 4.40%
Fig. 03-02 · Sample per arm against liftlog scale · lift 2–50%
Rule
n = (z₁₋α/₂ + z₁₋β)² · [p₁(1−p₁) + p₂(1−p₂)] / (p₂ − p₁)²
n = (1.960 + 0.842)² · [0.0400·0.9600 + 0.0440·0.9560] / 0.0040² = 39,473