One of the clearest ways to understand MarketDial is to walk through the testing process.
A convenience chain wants to test a slightly fancier-than-normal coffee machine to increase sales in the coffee segment of the business. These machines are expensive. They’re about $10,000 per machine, so they want to test whether this change will increase coffee revenue before they decide to buy machines for all their stores.
It sounds simple, but without a test-and-learn tool, it’s a difficult question to answer. Sales can easily be affected by unplanned noise in the data, such as weather, seasonality, a competitor closing down the street, or a holiday falling on a different day of the week, which makes the coffee machine test unreliable. Isolating the effect of one change from all that noise is the key, and what MarketDial is built to do.
Setting Up a Test
The convenience store chain is testing which stores get the new coffee machines. You can hand-pick them, test entire markets, or MarketDial can recommend the best set of stores for a clean read. These details feed into how MarketDial designs a test that can answer the question with confidence.
Sizing the Test: How Much Is Enough?
A test needs enough stores and enough time to be trustworthy, but not so much that it becomes slow or expensive. This is typically where teams get stuck without a test-and-learn platform like MarketDial. For the coffee machine test, we need to determine the right size and duration. Too small of a test, and it’s impossible to separate a real effect from random noise. Too large, and time and money are spent proving something the data already showed.
The platform shows, in real time, how confidence in the result changes as more stores or more time are added to the test.
Choosing Treatment Stores and Why It Matters
Next, MarketDial needs to decide which stores will actually receive the new coffee machines.
Whatever stores are chosen need to look like the broader convenience store chain they’re meant to represent. Remember, a biased sample gives biased answers. If the total fleet of stores is 30% urban and 70% rural, but the treatment stores end up mostly urban, the result won’t generalize. It’ll just describe what happens in cities.
So MarketDial builds a treatment sample that mirrors the full rollout group across dozens of relevant characteristics. Users can see and adjust exactly how representative the sample is, swap stores in or out intuitively, and see why any given store was excluded.
For the coffee machine test, MarketDial determined they would need 60 stores to be confident in the results of this test.
Choosing Control Stores
Once the treatment stores are found, MarketDial then matches each treatment store to control stores (those not receiving coffee machines), primarily based on how closely their historical sales trends track each other. The stronger the historical correlation, the more confidently any difference observed during the test can be attributed to the coffee machines, rather than to stores that were already trending differently. Users can exclude specific stores manually, adjust how many controls are matched to each treatment store, and see the trade-off in real time.
Now the coffee machine test is built. Let’s look at the results.
Test and Learn Results
From this test, the convenience store chain saw an almost 5% increase in coffee category revenue, with very high confidence. Confidence here is statistical significance; how sure the team can be that the change was actually driven by the coffee machines, rather than random noise in the data. For a high-stakes investment like this one, teams generally want to see at least 90% confidence before acting on the result. This test had 99.99% confidence.
What MarketDial Handles For You
That’s the full lifecycle of test and learn: a clear hypothesis, a statistically sound treatment and control stores, results backed by data scientists, and the capability to dig deeper into the data. MarketDial handles the statistical heavy lifting, sample sizing, store matching, outlier detection so retailers can spend less time on the mechanics and more time on the decision.
MarketDial is built so that anyone on the team can set up a test like this one, no statistics background required while still supporting deeper, more complex analysis for teams with a data science background, rather than forcing those computations to be done by hand. Sample sizing, store matching, outlier detection: MarketDial handles all of it, so retailers spend less time on the mechanics and more time on the decision
Behind the scenes, MarketDial runs the correct statistical calculations, monitored and continually improved by MarketDial’s data science team, so the answer that comes back is one that can be trusted. That data science team also acts as a consultant throughout the process, making sure the question being asked is well defined and that the test being built is actually capable of answering it with confidence.
Mitigating risk and optimizing opportunity with in-store testing
Enhance decision making with test and learn software
