Not all qual is equal
You’ve been here before. Your boss or client says, “have you spoken to the target audience, we need insights, stat,” and you…scramble.
The above scenario is one that we’ve been called in to fix many times before, because getting qual data isn’t as easy as it sounds. It all starts with butts-in-seats.
Butts in Seats
One of the biggest compromises I’ve seen in qualitative research is cutting corners on the people taking part. They provide the insight your campaign will be built on, so you need to speak to your exact target audience.
Sure, you might be able to speak to some customers, but are they the right customers? We got a frantic call from an agency. They had conducted a series of mini-groups with customers they had recruited from a Facebook page. Finding enough people wasn’t the challenge. In fact, a ton of customers wanted to take part. The problem was that the customers hanging out on the company Facebook page were very different from the customers who would be a fit for this type of product, and as a result they didn’t have anything to say about the topic at hand. Not all customers are the same, and good qual recruits to the exact target audience.
So, before the study even starts, we’re asking questions to make sure we’re speaking to the right customer profile. And that takes time, money, and a lot of telling the wrong kind of customer no.
Showing Up
This one is terrifying. 10 people have said they’d participate in your group…and only 2 show up. It happens all the time.
Sometimes it’s easier to not do something than it is to do it. And boy does this apply to qual research. Unless you can put the time into reminding and chasing down participants, this is best hired out. Trust me, it makes ALL the difference. You don’t have the time and energy to do this, but we have partners that do.
No isn’t Always a No
People say no about things that they shouldn’t be doing. Several years ago we did a study where I drove around the state conducting focus groups with school kids in rural parts of Oklahoma, uncovering what these teens knew about partying around abandoned oil sites.
We’d solved the problem of getting the right kind of kids in the group, but the challenge here was getting them to share about things they knew they shouldn’t be doing. On the surface, these kids said they didn’t party or go to these locations. But, after some light connection work (an agency guy in the room told the kids I was the voice of the Geico Gecko, and that went a long way), the kids slowly started telling stories about things they’d heard at school, which then turned into things their friends had done, which then turned into things they had done themselves.
Having a British accent helps too, because people find it naturally interesting. But one of my greatest tools is feigning ignorance. “I’m not from around here, can you explain this to me?” gets people talking.
If we had taken all those no’s at face value, we would have concluded that the problem our client thought they had, kids playing at well sites, just didn’t happen. But it did, and a lot of kids were doing it.
My point here is that sometimes questions have a baked-in socially acceptable answer, but it isn’t the real answer. You have to have a lot of confidence in your ability to work the questions and tell a real no from a soft no, and you have to be willing to challenge respondents without being confrontational. That takes time, effort and training, and it’s something we’ve spent all of our careers honing.
And speaking of experience…
Question Lists Suck
One of the worst groups I ever saw was run by another researcher. The researcher was personable, thorough, and great in front of a group. But the thing was, he treated the moderator’s guide, or the question list, as a static thing, much like a quant survey.
So much insight was lost because the order of the questions on the guide didn’t match the natural flow of the conversation. One of the most beautiful things about qual research is that it is about the art of conversation, not about the art of reading questions from a list. Pivoting, jumping on something someone has said in the moment is a key part of gathering intelligence.
You see this most in discovery work, like new product development, where we’re going into a market that we and our clients know little about. We’ve seen it in crypto and NFT interviews, tech and fintech. A respondent says something you’ve never heard of, or something you didn’t think was a big deal and it turns out it was, and it’s your job to explore it thoroughly. Those interviews often go off the guide, but in the best possible way, because we’re truly discovering.
One of our favorite clients is a talented team of corporate researchers and I LOVE their approach to qual. We get given the business objectives, and as long as our groups or interviews speak to those objectives then we can ask the questions in any way and any order we like. It’s such a liberating and brilliant way to get to the matter at hand.
We’ve run research where our guide for a 45-minute interview was four goals on a sheet of paper, with no questions at all. It was just as good as having three or four pages of questions, because our job was to weave a conversation around those four goals.
Counting Heads Misses the Point
This last one is about what happens once the groups are done. I’ve seen unskilled interviewers, and unskilled analysts too, treat a focus group like it’s a quant survey. So they’ll report that 7 out of 8 people said this, or 3 out of 5 people said that, and layer on this quasi-statistical element to support their findings.
That’s incredibly dangerous, because as we’ve seen time and time again in qualitative research, it just takes one person to have a spark of an idea. One person says something that changes the scales of the entire conversation, and that’s where the qualitative gold exists. It never exists in this kind of weird statistical voting, and “4 out of 5 people said this” is never a smart way to report qual findings.
Don’t get me wrong, there is value in saying 16 out of 24 focus group respondents preferred “concept A”, but the magic is finding out why they preferred it and not the other concepts. And that’s kind of the point about qual.
So, the next time your boss or client says “we need insights, stat,” you don’t have to scramble.
If you’ve got interviews or focus groups coming up, drop me a line and let’s talk about how we can help.
Next in the series: The power of qual and quant.
Check out the additional Unsaid articles here:
Part 1 of 4: It’s about what’s unsaid
Part 2 of 4: What qual can do that quant can’t
Posted by
-
Kevin JessopCEO & Research Director