Get Access to our Extensive Library of A/B Testing Content and Resources

Implications of Multiple Control Matches on A/B Testing Accuracy and Frequency

We conducted a simulation experiment to examine the effect of multiple control matches on A/B testing accuracy and frequency. Our simulation experiment shows that 1-to-many matching offered slight improvements in noise reduction when compared with 1-to-1 matching. However, the improvements are not substantial enough to warrant a decrease in testing frequency, representativeness, and comparability that occurs in 1-to-many matching conditions. We recommend 1-to-1 matching as the best method to optimize return on investment for A/B retail testing. We conclude with recommendations for retailers and future researchers.

Get Access Now

Fill out the form to read more and find out how A/B testing can help drive sales and grow your business.

Give MarketDial a try.