Statistical Significance

A measure of confidence that the result of an experiment is real and not due to random variation. Typically set at 95% confidence in marketing experiments. Calling a test winner too early is the most common A/B testing mistake, especially with small sample sizes.