Product Development

Without Free-Choice Analysis, You Don’t Know What Consumers Want

If you want to understand how consumers perceive products, using free-choice profiling gives you a more accurate picture than closed questions.

Let’s say I want to get your opinion on a new chocolate bar I’ve created. I tell you the flavors in the chocolate are “sweet,” “raspberry,” and “caramel.” I ask you to rate how much you like each flavor out of ten. 10/10 means you love the flavor; 1/10 means you hate it.

You give the chocolate a try, but you don’t taste any raspberry. To you, the chocolate tastes like coconut. Too bad. You’re not allowed to list a new flavor, and you have to rate your preference for the three flavors I’ve given you regardless. You end up rating the non-existent raspberry flavor as 3/10.


You’ve got to rate the flavors I’ve listed, and you can’t add any more to the list

Imagine if I conducted this survey with hundreds of people, and I forced each one to rate the raspberry flavor, whether they tasted it or not. I’d end up with some deceptive data. Seems like people really hate raspberry, huh?

That’s the problem with closed questions in CPG product testing.

So, what’s the alternative?

In free-choice profiling, panelists are able to list what they taste rather than choose from pre-selected attributes. You’d be able to tell me: “I like this chocolate because it’s creamy, it has a strong coco-nutty flavor, and a caramel aftertaste.

The problem is, that kind of description is pretty hard to plot onto a graph. It also might be difficult for you to define the flavors on your own.

Let’s say you taste the chocolate bar and detect a hint of aniseed. You love it. This aniseed-caramel balance is so perfect you think this might be the best chocolate bar you’ve ever tasted. Of course, you can’t tell me that, because you don’t know the word for aniseed.

If you want to understand how consumers perceive products, using free-choice profiling gives you a more accurate picture than closed questions. But it's difficult to draw statistical insights from free-choice descriptions. Our AI system allows you to get the advantages of free-choice profiling without the drawbacks.

Why Closed Questions in Consumer Testing Don’t Work

To use closed questions in consumer testing, the products first go through a round with professional tasters. The reference-trained expert-tasters conduct ‘descriptive analysis’ — they define the flavor attributes that are in the product. Then, the product goes to a consumer panel, and they rate how much they like those flavor attributes — this is ‘preference analysis.’

A disjointed process like this is problematic because it separates preference from perception. You might get accurate data on whether consumers like or dislike your product, but you don't really know why people like or dislike it.

Another issue is that those professional tasters don’t represent the normal population. Jason Cohen, the CEO, and founder of Gastrograph AI, points out that trained expert tasters have a “descriptive vocabulary [that] is significantly different from that of consumers in frequency and meaning.” So the results you get from this kind of testing are “skewed, misleading, and frequently wrong.”


If you don’t ask consumers what they taste, the data you gather is incomplete

Closed questions are often used with ‘forced responses’ — that’s me saying YOU MUST rate the raspberry flavor, regardless of whether you taste it. When tasters aren't free to list what they taste, they end up ‘offloading’ flavors. Say you taste something super acidic. You’ve no way to record this flavor, so you either don’t communicate it, or you offload it onto the raspberry and give the raspberry flavor a low score.

As well as misleading data, you also get incomplete data because panelists aren’t able to communicate everything they taste. You’re not “using the complete range of each panelist’s experiences and the terms they would use to communicate those experiences,” says Jason.

Why Conventional Free-Choice Profiling Isn’t Enough

Free-choice profiling gives you a truer picture of what an individual consumer tastes than closed questions. But it doesn’t give you responses you can pull useful insights from.

When you let tasters describe products freely, you get rich, descriptive data. Descriptions from free-choice profiling tell you a lot about what an individual thinks of a product. But the language isn’t standardized — we might taste the same flavor, but I’ll describe it as “coffee,” and you’ll describe it as “bitter.” So you’re not able to easily spot patterns or draw insights.

Humans are good at perceiving flavor but notoriously bad at describing it. “If someone hands you a fruit-flavored candy, and they don’t tell you what the fruit is, it is really hard to identify,” explains Jason. “If they then give you a narrow list, say 20 fruits, and they say it’s one of the fruits on this list, your identification rate increases. You are seven times more likely to pick the right fruit from a list than if you’re not given a list…Everyone knows what they’re perceiving. It’s an identification problem.”

Another example is the fact that many people commonly misattribute the flavor of bergamot. Because there’s bergamot in Earl Grey tea, many consumers believe bergamot to be the flavor of Earl Grey. When they taste something tea-like, they’ll say it’s bergamot. But bergamot is actually citrus.


Earl Grey tea is made with oil from the rind of bergamot oranges

Our ability to accurately describe flavors depends on our experiences. Since most of us don’t have a background in food technology, we’re usually pretty bad at putting names to flavors. “That makes it really, really difficult to figure out what someone is talking about,” says Jason.

Advanced Free-Choice Profiling With Gastrograph AI

We’ve trained our AI models on data from thousands of tasters and use natural language processing (NLP) to analyze what ordinary consumers mean when they describe flavors. When you use Google search, it’s NLP that allows the search engine to understand how words are used in context so it can make sense of searches and deliver useful results. At Gastrograph, NLP allows us to interpret what flavors consumer tasters are perceiving based on what they say. That means people are free to describe flavors in a way that makes sense to them, and we can still get accurate, useful insights from the data.

On the Gastrograph platform, there are 24 flavor attributes (for example: rich, dry, bitter, fruits). Within those 24 attributes, tasters input whatever word they want (in any language they want!) to describe what they taste. Let’s say I want to record the flavors in a soda I’m tasting. I might select ‘Fruits,’ and then describe the flavor by entering “strawberry-lime.”


After choosing one of the 24 attributes, tasters can select a label listed on the platform or enter their own word to describe the flavor

“The 24 attributes that we've identified totally encompass the gustatory flavor space. So there are no perceptual stimuli that an individual can taste in the normal course of a food or beverage tasting that doesn't have a place on the graph.” – Jason Cohen

The AI system has learned to understand all the different ways people describe flavor. Based on the attributes they select, it knows what they are tasting, regardless of whether they describe it accurately or not.

“We're able to use [the tasters'] descriptive analysis to understand what they think that they're tasting, and we're able to use where they fall in flavor space to understand what they're actually tasting,” explains Jason. Tasters can add as many flavors as they want, and there are no forced responses. You’ll not find any flavor offloading around here, just detailed and accurate data.

It’s a bit like if you’re trying to get Alexa to play a song. Maybe you can’t precisely remember the name of the artist, or maybe you pronounce it wrong. Alexa responds and either tries to narrow down what you’re asking for or guesses what you mean.

For example:

“Alexa, play Elton Jobs.”

“...did you mean Elton John?”

And before you know it, you’re singing along to Tiny Dancer at the top of your voice.

The fact that tasters go through the process of identifying flavors in the system also means the predictions Gastrograph makes are indicative of sustained preference. Tasters have had to think about the flavors more than if they were pre-defined for them, so the results are more indicative of a two- to three-week tasting trial.

Step Into the New World of Data-Backed Product Development

Consumer-led flavor profiling is one way Gastrograph AI can help companies to make better product decisions. Learn more about the possibilities of AI for new product development.


Similar posts

Stay up to date with Gastrograph AI

Be the first to know about company updates and industry-related news, straight from our internal subject matter experts.