Michelle Niedziela, PhD, HCD research Trust Me: Quality Control in Neuromarketing Research

Dr. Michelle Niedziela, PhD © Niedziela

Von Michelle Niedziela, PhD

At the beginning of this year, SPIEGEL Online shook the world of Market Research with a damning article: “Manipulation in market research: How surveys are forged and customers are cheated”. The article revealed internal documents leaked from various (popular and large) market research companies proving repeated and deliberate research fraud. The range of deceptive market research studies spanned from fake interviews to fake respondents, bringing into question even the most trusted forms of traditional market research.

Qualitative and quantitative market research methodologies, such as interviews and surveys, are the backbone of strategic business strategy. Getting the voice of the consumer is one of the key factors used in maintaining competitiveness against rival brands. Being able to trust in contracted research results is paramount to healthy, functioning industry research and often this is entrusted to research partners. For companies to obtain that consumer voice they frequently rely on market and consumer research providers. And by relying on a third party to provide that product, they are relying on a basic level of trust in that they are getting what they paid for. On an already existing background of mistrust in surveys (i.e. that they don’t tell the true voice of the consumer), the SPIEGEL article brought some confirmations to those arguing about the accuracy and quality of the data collected via market research. The SPIEGEL investigation is the latest to put a spotlight on this growing threat.

"Clients push for CHEAPER, BETTER, FASTER"

Clients push for CHEAPER, BETTER, FASTER. But unfortunately, that simply cannot exist and it does take fortitude to push back and say no as a provider. It is possible to provide research that is GOOD and FAST, but it won’t come cheap. It’s possible to provide GOOD research CHEAP, but it won’t happen fast.  It’s also possible to provide FAST research CHEAP, but it won’t likely be very good. Yet the threat is real that if you can’t provide all three, potential clients will simply go elsewhere to someone who says they can.

© Niedziela

And therein lies the problem. When an ethical provider pushes back, the client can just as easily take their business elsewhere which can often unwittingly lead them down the path to a potentially shady provider. And if they don’t push back, their work may suffer by being more prone to mistakes.

Of Neurohype and Neuroethics

The problem of better/faster/cheaper has become even more of an issue with innovations and advances in technology used in consumer research such as neuromarketing (also known as applied neuroscience, consumer neuroscience, etc.). But does the product meet the promise?

Neuromarketing uses neuropsychological methodologies from academia to study consumers' sensorimotor, cognitive, and affective response to marketing and product stimuli. The idea of using neuromarketing over traditional market research is an appealing one, especially given the current climate of mistrust in the traditional measures. Further, neuromarketing companies have taken to suggesting that not only can you not trust traditional market research providers, but that you also cannot trust consumers to tell you the truth. Such that when answering a survey, consumers may just be telling you what you want to hear as opposed to how they truly feel. Neuromarketing companies suggest that by using methodologies from neuroscience and psychology, you can bypass the consumer voice and go straight to their unfiltered perceptions. Or even further, neuromarketers imply that neuromarketing, because it often uses physiological data or implicit reactions, is somehow unbiased.

“Face the evidence: customers don’t know what they want! Traditional research methods won’t help you find what triggers decisions in their brains because they are based on self -reports.” (neuromarketing company website)

“We create solutions to better understand consumers’ true emotions, motives and drivers.”
(neuromarketing company website)

I’d argue that this line of sales pitch is puffery at its best and dangerous at its worst. From a research provider perspective the danger is that in applying these methodologies incorrectly, we poison the well, so to speak. And clients become disappointed and disillusioned by their poor experiences with the theories and technologies.

Neuromarketing: What is it good for?

When I left academia and entered industry, my first job was to act as the scientific lead for external research/innovation at a large CPG company where it was my job to work with external research providers using what is now referred to as consumer neuroscience but back then didn’t really have a name.

Having now worked on both sides of the relationship, it has really pained me to see how this field has developed. What started out as an interesting concept has since exploded.

"According to the theories System 1 is the emotional decision maker that marketing should target for stronger appeal and ultimately drive consumer purchase."

Published in 2011, Kahneman’s book Thinking, Fast and Slow has become dogma to neuromarketers, dividing consumer decision making into two processes: System 1 and System 2. System 1 being the fast and emotional, reactive decision making process and System 2 being the slow and deliberate, purposeful decision making process (*this concept isn’t all Kahneman and in fact can likely be traced back to Plato’s Chariot Allegory or maybe even Freud’s Id, Ego & Super Ego). According to the theories System 1 is the emotional decision maker that marketing should target for stronger appeal and ultimately drive consumer purchase.

In fact, once upon a time, a neuromarketer tossed out a number: 95% of all purchasing decisions are made subconsciously.

It sounds great. But it’s total fiction. It seems to stem from the idea that we only use 10% of our brains for conscious thought and all else (90+%) is non-conscious ignoring the fact that our brains are busy maintaining body homeostasis. The stat is often credited to Martin Lindstrom in reference to mirror neurons, or sometimes to Dr. Gerald Zaltman (with no real agreement on who owns this number), however no actual evidence exists proving that this statement is true. Yet that has not stopped it from being one of the most cited, yet un-sourced, statistics in the field.

That’s not to say that System 1 style thinking is useless. In fact, a lot can be inferred from what activates System 1. Certainly there are sensations of which we are not consciously aware and sensations of which we are aware. For example, when your cell phone rings the hair cells in your ear react to sound vibrations and send a neural signal to the brain. This happens without you being consciously aware of the sound quite yet (non-conscious). But as your brain receives the signal, it classifies its meaning and value and then deliberates on that information to finally decide, consciously, how you will react. The utility of understanding the non-conscious actions (the sound of the cell phone) of this process is that through better understanding the non-conscious response we may be able to influence the conscious effort (answering or not). For example, changing the tone, pattern or length of the ring tone may influence how quickly you respond to it or changing the color of a package may influence perceptions of a product.

Using the right tool for the right question

There are quite a few options in the neuromarketer toolbox. Psychological and neuroscience tools can range from fMRI to psychological questionnaires. And this is where my manifesto becomes a bit more controversial and I address the misuse and bad science in the field. But I’d like to stress here that it isn’t the tools themselves that are bad. In fact, I like to say that the tools do exactly what they are supposed to do; it’s humans which are more often the problem by over-interpreting results or designing studies incorrectly, and yes, adding bias (experimenter and confirmation bias).

The Gold Standard (biometrics measures)

fEMG (facial electromyography), HRV (hear rate variability), GSR (galvanic skin response) are considered the gold standard mainly due to their simplicity and direct correlation to what they measure. For example, increases in GSR are directly and positively correlated to increases in arousal. However it can be limiting in that you are restricted to central location testing in person and low sample sizes as testing requires as higher level of environmental control. While technologies for online, webcam based measurement of biometrics does exist, that data is of much lower quality with high amounts of variability and environmental noise.

•   Eye Tracking

Eye Tracking is a direct measure of gaze behavior and implicit reaction measures are directly correlated with association. This is a great measure for analyzing where consumers are looking, however it is often mis- or over-interpreted. For example, far too often eye tracking behavior is attributed to attention when in fact it is possible to be looking at something but not paying attention.

• Implicit Association Test (IAT; Implicit Response)

While easy and cheap to run, improper design in implicit reaction studies that make their results less reliable.

• Electroencephalography (EEG)

EEG measures can range from the more expensive and involved EEG headsets to cheaper and less reliable headsets. Cheaper EEG headsets mean a poorer signal, meaning more difficulty in interpreting already difficult to interpret results.

•  Functional Magnetic Response Imagery (fMRI)

fMRI is a great tool for associating different behaviors with specific brain structures. However, due to its high cost and need for hospital settings it is not feasible for most industry research. Further, interpreting results can become quite complex and proper research design can be quite limiting. fMRI studies are notoriously expensive and difficult to perform in the confines of consumer research. Extrapolating emotional conclusions from EEG or fMRI work is not as simple as it may seem and typically requires evoking the reactions, not passive measurement. However, this step has often been skipped in industry use, making the conclusions hazy at best and total false at worst.

• Facial Coding

Facial coding may not be as useful as it is sold, as its proponents often neglect to reveal its limitations such as socially driven reactions, dropout rates, and problems in interpretations.

"So while some research providers claim that you can’t trust consumers to tell you what they really think, I don’t agree that that is necessarily true, although it makes for a very convenient story"

Ultimately, there is no one tool that will cover all research. And so we must be willing to accept that certain tools are better at collecting certain types of information over other tools. And that we must be sure we are using the right tool for the right measure. It is perfectly reasonable to use any one of these measures as long as you are clear on all the limitations AND use them properly.

How do we fix a problem like Neuromarketing?

While it’s great to use all of these scientific tools and be on the cutting edge of technology, it’s important to take a step back and think about what you are ultimately trying to accomplish. It’s my firm belief that if you can just ask someone, then just ask them. If the question is about liking, for example, you are much better off simply asking consumers if they like the product. Consumers are actually quite reliable at knowing whether they will purchase something or if they like something. So really, that’s not what the technology should be used for and is in fact, not great at doing.

"For example, if you research question is about whether a new fragrance helps to suggest that the product is more "spiritual", facial coding will not be able to help you. But implicit reaction may be able to help."

So while some research providers claim that you can’t trust consumers to tell you what they really think, I don’t agree that that is necessarily true, although it makes for a very convenient story. The truth is that consumer can tell you what they think if you ask them correctly and neuroscience really isn’t a great tool for lie detection (except for pupil dilation which has some reliable correlation to lying).

So what can you do?

I suggest following a few rules/guides to help decide how to use both neuroscience and a potential neuroscience provider:

1.   Start with the research question.

While it is often attractive to passively measure consumers in a naturalistic environment and there certain can be exploratory ideas uncovered in observational research, in order to get the best and most actionable results for applied neuroscience you should consider what the research question is. Starting with the research question will help guide the scope of the study and the approach of the method.

2.   Always use the right tool for the right question.

Once you have the right question, it can be a lot easier to choose which research tool will best provide an answer. This is a much more productive and cost efficient approach than starting with a tool and looking to apply it somehow. For example, if you research question is about whether a new fragrance helps to suggest that the product is more "spiritual", facial coding will not be able to help you. But implicit reaction may be able to help.

And this is why it is important that you research provider be "methodologically agnostic". Or as I often say, if you go to a widget salesmen he is going to sell you a widget and not something else. If the research provider is a "one trick pony", or only has 1 methodology to offer you, he likely won’t be trying to tell you about the limitations of that method.

So how can you identify a good research provider? I suggest asking a few targeted questions about the limitations of the proposed methodologies. And if they can’t tell you about any limitations or suggest that there is one solution to fit all needs, then likely they are not a great research partner.

3.     Build a story with multiple research points

Neuromarketers often try to say that consumers’ cognitive responses can’t be trusted and that neuro measures are somehow more truthful. Neuro measures are not a replacement for cognitive or more traditional measures, they don’t answer the same questions. Instead of trying to make neuroscience data stand alone, supplement current research with additional insights from neuroscience and psychology. By integrating the data (either in story or through statistical modeling) it is possible to make better and more actionable, informed conclusions and interpretations.

"At a neuromarketing conference, I once witnessed a research provider being called out. An audience member asked him how he had validated his neuro-methodology. And his shocking response was "that’s not my job".

At a neuromarketing conference, I once witnessed a research provider being called out. An audience member asked him how he had validated his neuro-methodology. And his shocking response was "that’s not my job".

It is the job of the research provider to use reliable, validated methods and technologies. The client-provider relationship is one of trust. And so we must do our very best to nurture that trust with full disclosure regarding the limitations of these tools. While it is important to always push the limits and create new and innovative applications, we must most importantly stay scientifically vigilant and maintain scientific integrity.

About the Author

Michelle Murphy Niedziela (PhD; @hcdneuroscience) is a behavioral neuroscience expert in neuropsychology, psychology and consumer science. As VP of Research and Innovation at HCD Research, Michelle focuses on integrating applied consumer neuroscience tools with traditional methods used to measure consumer response.