Leading producer Cutwater Spirits ran their first national Super Bowl ad in 2022. We wanted to see how consumers feel about the product, so we...
The Golden Days of Central Location Tests Are Over
Poor product testing results in failed product launches. If you want to launch successful products, it’s risky to base your product decisions on CLTs. Our AI system lets you bring the consumer's voice into your product development in a reliable, scientific, and cost-effective way.
The mid-20th century was an exciting time for market research. Arthur Nielsen founded one of the first-ever market research companies, Ernest Ditcher pioneered the focus group research method, and companies started to interview real-life consumers. In the late 60s, there was a move toward large-scale consumer testing, and “voila, a new era of data collection — central location testing — was born.”
Since the 1970s, not a lot has changed. Central Location Tests (CLTs) are still considered the holy grail of customer research. After all, what’s better than bringing in the voice of the consumer? And what’s more convenient than getting a bunch of people to test your product on a single day in a single location?
Data-driven decisions are great. But not if you're using bad data.
Poor product testing results in failed product launches. If you want to launch successful products, it’s risky to base your product decisions on CLTs. Our AI system lets you bring the voice of the consumer into your product development in a reliable, scientific, and cost-effective way.
What is a CLT?
In a CLT, a large group of consumers tests your product in one location — typically a shopping center or large hall. CLTs usually take place in one day, unlike at-home tests where consumers try a product over an extended period of time.
To carry out a CLT, you need to find a suitable sample group of consumers and trained interviewers to do the testing. All of this planning and organization is time-consuming and expensive.
What Are the Risks of CLTs?
When you’re making big-money decisions (for example: shall we pay for this product launch or not?), you want to base your choice on hard data, not guesswork.
You want evidence, not assumptions.
Unfortunately, the flaws in CLTs and how they're used by most CPG companies give you incomplete or questionable data — so you’re forced to guess.
Ideally, CPG companies would test their product with several demographics at all stages of product development. But the time and expense of CLTs mean that that's not possible for most companies. They’re a reserved resource. Brands have to select the tests they do and make some tricky decisions.
Where do you test?
There’s a lot of variation in consumer preference even within individual countries. For example, there's a difference in flavor preference between north Italy, south Italy, and Sicily. There's also variation “between east Germany and west Germany still remaining to this day,” notes Jason Cohen, CEO and founder of Gastrograph AI.
If you want to launch one product for a group of countries or for a whole continent, you need to test each subpopulation for accurate results. That amount of testing is expensive and practically impossible.
So you might end up conducting a single test or a handful of tests and then trying to apply that data to the whole region. But data you collect in one location isn’t representative of customer preferences in other areas.
When do you test?
You’ve also got to make some hard decisions about the stage at which you do your testing:
Do you test at the start of the development process so you can create a product to meet consumer preference?
Or do you test at the end so you can check that the ready-to-launch iteration of your product is likable?
Either way, unless you’re prepared to pay for tests at every stage, the voice of the customer isn’t there throughout to inform your decisions. Companies that exclude customers from their development process risk creating products that don’t meet core consumer needs.
Who do you test?
It’s difficult to build a sample that represents every demographic: you have to decide who to include in your sample — and who not to include. You might choose based on your assumptions. For example, you think your product will be popular with older demographics, so you create a sample of people aged 65+.
But if you do that, you don’t explore all possibilities. Perhaps teenagers would love your product, but you’ll never find out because you never included them in your sample.
Most companies make the mistake of targeting people who already use their product several times a week.
“Targeting heavy users is one of the dumbest things that companies have ever decided to do.” — Jason Cohen
‘Heavy users’ consume more of a product than the general population, so they don’t represent the preferences of an average consumer. Targeting heavy users is especially problematic in declining industries. If you love a generally unpopular product, then your preferences are different from the majority of the population.
Take, for example, soda, which is in decline. “[Soda companies] recruit their heavy users, the people who still drink soda, who are becoming more and more different than the rest of the population. And they produce sodas that sell well but sell less. They're just managing a steady decline,” says Jason.
When companies do this, they’re “not bringing new people into the category or into the brand to capture the next generation of drinkers.” They’re also missing opportunities.
In the last couple of decades, sparkling seltzers and flavored water without sweeteners have gained a lot of popularity, “but soda companies didn't see it,” emphasizes Jason. Instead, they focused their product development around the increasingly rare people who still drink soda.
Disjointed Data Sets
In CLTs, you typically have a bunch of expert tasters who do descriptive analysis and the consumers who do preference analysis. You treat people either as sensors (who tell you what they perceive) or as consumers (who tell you what they like), but not both. Jason refers to this as the central dogma of sensory science and, in his words, "the central dogma of sensory science is incredibly dumb."
People aren't that good as sensors. "Humans are very opinionated, and they're going to find ways to insert their opinions into your results," explains Jason.
If you separate descriptive analysis from preference analysis, you separate perception from preference. So you end up with disjointed data sets — one set from the expert tasters identifying the flavors and another from consumers rating those flavors.
When you don't ask consumers what they perceive, only what they prefer, you can't accurately identify why they like or dislike a product. Maybe their liking or disliking for a product is based on a flavor the expert tasters didn't list, and they don't have any way of communicating this to you. Data gathered from testing in this way doesn't give you an accurate or complete picture of consumer preference.
To get statistically useful data from CLTs, you ask very narrow questions — for example, asking consumers to rate a specific element of one product. Because these questions are so specific, the responses are only relevant to that narrow query. You don’t get a complete picture of how consumers perceive flavor, and if you want more information, you have to pay for more tests.
Consumer preference not only varies a lot, but it also changes over time. So your results are only valid for the demographics you’ve tested and have an expiry date.
After a while, you have to throw the data away and test all over again.
What’s the Alternative?
Gastrograph AI takes consumer testing from a place of scarcity to a place of abundance. We’re able to use AI models (built on data from in-person tastings) to make thousands of predictions in the time that it would take you to do one CLT. It’s the equivalent of being able to test each version of your product everywhere, all the time — so you eliminate the risky who/where/when decisions.
When you can run the equivalent of 100 tests as easily as one, you can develop iteratively. For example, you could adjust the salt level by 1% each time to find the perfect level for your target demographic.
It also takes the pressure off testing. You’re able to be creative and explore new flavor combinations without paying tens of thousands of dollars for every single experiment.
We’re able to tell you which version of your product would be popular with the largest percentage of the population and suggest changes you should make to increase likability.
PQ stands for Perceived Quality — a score based on how much we predict consumers will enjoy your product, using a 7-point hedonic scale
On the Gastrograph AI platform, consumers are free to describe flavors in whichever way they want, and our system will make sense of it. Tasting data gathered in this way gives an accurate picture of how someone perceives a flavor, so we can see which aspects of your product their liking comes from.
We continuously collect tasting data all over the world to update our system and model how preference changes over time. That means our data is reliable, and our predictions are always up-to-date.
It’s Hard to Let Go of Tradition
In the 1970s, CLTs were cutting edge. But the world has changed. It’s time to update your product development process so your company can step into the 21st century.
If you’re willing to be a CPG pioneer, you can start using data to make smart product decisions. You’ll be able to develop innovative, exciting products that consumers will love. Read more about the future of product development.