Jerry Rice, perhaps the greatest football player of all time, didn’t achieve excellence by happenstance. Known for his maniacal work ethic, Jerry Rice said, “Today I do what others won’t, so tomorrow I can do what others can’t.” Rice’s thinking applies perfectly to store testing. For a retailer to outperform its competitors and survive in the modern environment, it must operate differently, working not only harder but smarter too.
This is where A/B store testing comes in. Savvy retailers will use this opportunity to better their test-and-learn practices, experimenting with new designs, promotions, and click-to-brick initiatives. And these retailers, the ones who are doing today what others won’t, are the ones that will succeed tomorrow where others can’t.
Why test?
Retail experimentation is not only possible, but crucial. However, the current environment is challenging as consumer behavior changes rapidly and many retailers lack execution capabilities. To help, the data-scientists from MarketDial have put together the following pointers:
- Test innovative hypotheses rapidly
Consumer behavior is changing rapidly. Now-common terms like “contactless delivery” didn’t exist until the pandemic, curbside delivery used to be a niche market, and if a masked man entered your convenience store before 2020 you’d be emptying your registers instead of selling him groceries.
This unique consumer behavior presents a challenge: how do you test and learn valuable insight in a market that changes so quickly? The answer is to test as quickly!
To respond to fluctuating consumer behavior, retailers must design short-duration tests and implement their initiatives rapidly. In doing so, retailers can match the evolution of consumer behavior. Agile retailers will view this as an opportunity to cater to their consumers where their competitors either cannot or do not.
By adopting a level of agility in testing and implementation, a retailer can learn whether to adopt curbside delivery, extended or shortened hours, self check-out, or click-to-brick marketing. Store testing will give a retailer the confidence it needs to invest behind profitable initiatives and to avoid unprofitable ones.
Be careful though, short-duration tests often require either large test and control groups or they generate low confidence. Fortunately, MarketDial’s platform leverages machine learning to create higher confidence tests in shorter durations and with fewer stores, allowing retailers to test at a pace that matches consumer-behavior evolution.
- Test resilient initiatives
In addition to testing current-event driven (such as pandemic or inflation) hypotheses, retailers should use this time to test hypotheses that are more resilient, less affected by the external fluctuations, and that will remain applicable after the external factors resolve. Unlike your environment-specific initiatives, these hypotheses can be tested over longer durations. For example, hypotheses related to product bundling, macro-space, store layout, or online purchase and in-store pickup are applicable now and the results of these tests will guide retailers in a post-COVID, post-inflation world. Remember, even if consumer behavior is unusual now, experiments can still yield valuable insights because quality control-store selection and measurement will reduce the noise and ensure statistical significance.
When others begin to experiment again, retailers who have validated their hypothesis during this pandemic will be ahead of their competition, able to implement incremental improvements on their business that could be the difference between fortune or folly.
- Use proper test design and measurement
Store testing with proper design and measurement is always critical. But with the current noise in the market, retailers now have little room for error. With proper test design retailers can get actionable insight from their tests, even during the pandemic. Good test design includes: (a) a clear hypothesis, (b) representative test-store selection, (c) proper control-store selection, and (d) accurate lift measurement.
Develop a clear hypothesis
A clear hypothesis focuses your test toward actionable insight. A quality hypothesis is clear, measurable, falsifiable, and discrete–meaning it measures only one outcome at a time. A hypothesis leads to accurate test design because it forces thoughtfulness in the purpose of the test, where the test will run, how a retailer will quantify success, and how a retailer will eventually execute the initiative if successful.
Select representative test stores
Test-store selection is critical. Selecting a biased sample could lead to a false negative or, potentially worse, a false positive. Selecting a representative sample involves measuring store differences across various attributes (e.g., volume, size, product mix, customer profile, area income, competitive density) and ensuring your sample measures similarly across these attributes. By selecting proper test stores, a retailer ensures that the measured impact can be leveraged to create an accurate prediction of how the initiative will work on implementation.
Select comparable control stores
Control-store selection is equally important. A well-matched control group ensures a baseline to measure against the test group. To select proper control stores, you must ensure that your control stores have similar seasonal sales, volume patterns, store attributes, and customer profiles. In doing so, you can accurately measure the effect of the initiative.
Use accurate lift measurement
Lift is measured by comparing the change in sales (or any other variable like traffic, transactions, or profit) before and after you implement the test for both the treatment and control group, and netting those differences. So if sales increased in the test group by $1,000 per store and in the control group by $400, then your lift would be an incremental $600 per store.
Solid test design and measurement using the principles above will ensure you can measure any initiative properly, even through the noise of the pandemic.
Be like Jerry Rice, do the work now when others won’t
This season in retail history is a challenging yet opportune time to test, not because it is easy, but because right now you can improve while others don’t. You can test innovative ideas that will thrive during high-stress circumstances and you can test resilient ideas that will survive in a post-stress environment. But remember, proper test design is crucial as it enables you to reduce noise, test rapidly, and gain actionable insight.
For easy, accurate, and actionable store testing, contact us at MarketDial. To learn more about how test and learn can support your ever-changing retail needs, check out these resources:
What is test and learn?
Test and learn examples of retail success
A test and learn mindset: discover, adapt, and succeed
