CUPED explained: faster A/B tests with variance reduction
A practical explanation of CUPED for A/B testing: how variance reduction works, when it helps, and what to watch before trusting the results.
CUPED is a variance-reduction technique for online experiments. In plain terms, it uses what you knew about users before the experiment to explain some of the noise in what happened during the experiment. Less noise means you can often detect the same effect with fewer users or less time.
Why it works
Many experiment metrics are correlated with past behavior. Users who spent more last month may be more likely to spend this month. CUPED uses that pre-period signal as a covariate so the treatment effect is estimated on a cleaner outcome.
When it helps
- The pre-period metric is strongly correlated with the experiment metric.
- The covariate was measured before treatment assignment.
- The data is reliable and available for most users.
- The team understands the adjusted metric and reports it consistently.
What to watch
CUPED will not fix broken randomization, sample-ratio mismatch, peeking, or bad metric definitions. It is a powerful addition to a healthy experimentation process, not a replacement for one.
Used carefully, CUPED can make experimentation programs learn faster without lowering the evidence bar.