Not Everyone’s Easy to Find: The Truth About Sampling

Several yeas ago we took on a project in the cement industry where we were tasked to interview contractors about their purchasing decisions. While this might sound simple enough, a major wrench (or spanner for my UK friends) in the works is that contractors are on job sites all day. They’re covered in dust, suspicious of anyone pinging their phone, and they’re definitely not sitting at a computer waiting to take your survey. That project taught us more about recruitment than a dozen consumer studies ever could.

Sampling works like that — the audience defines the difficulty, and the difficulty defines the timeline, budget, and sometimes whether a project is even feasible.

“Everyone” Is Not a Target Audience

We get this a lot with new clients – when we ask “who do you want to talk to?” the answer comes back: “Well… everyone.”

Which, respectfully, is never the right answer and something we’ll always push back on.

Part of our job, and it’s an uncomfortable yet essential part, is helping clients get specific about who actually matters to their research question. Because when you try to talk to everyone, you end up with data from the people who were easiest to reach, and those people might have absolutely nothing useful to tell you.

The Recruitment Difficulty Spectrum

Some audiences you can recruit in 48 hours. General consumers? Online panels make that almost embarrassingly easy. You set your criteria, launch, and the completes roll in.

Then there’s the other end. C-level executives, for instance. We’ve run projects where a single 30-minute interview cost $1,200 per respondent (that’s our hard cost, by the way). You read that correctly. Senior decision-makers value their time at senior decision-maker rates, and if you want them to give you half an hour of honest insight, you’re paying for it. And even at that price point, you’re competing with every other research firm, consultancy, and journalist who also wants thirty minutes of their day.

And then there are the audiences that make even C-suite recruitment look straightforward. Through our work with clients in emerging and specialist industries, we’ve recruited YouTube content creators for tax compliance research — try explaining to an influencer why their accounting habits matter to a fintech company. Crypto traders and NFT investors for financial services studies, where half your potential respondents are pseudonymous by design. Oil and gas compliance officers who can’t discuss half of what they do without checking with legal first. State and municipal government officials who need three levels of approval before they can participate in anything.

None of these audiences recruit the same way, and the reasons people say no are just as varied as the channels you use to find them.

Stew wrote recently about research fatigue — the very real problem of people being asked to participate in too much research. That compounds with hard-to-reach audiences. They’re not just hard to find, they’re hard to convince.

The Right Recruiter Changes Everything

For specialist audiences, we partner with recruiters who genuinely know the space. Not generalists who’ll blast a LinkedIn message and hope for the best — people who have relationships, who understand the industry language, who know which conferences these people attend and which Slack communities they lurk in.

That expertise comes at a premium, and sometimes a steep one. But the alternative is three weeks of spinning your wheels and ending up with respondents who technically qualify but don’t really represent the people you need to hear from. Jenn’s written about how small changes in survey design can dramatically improve data quality — the same principle applies upstream. Get the recruitment wrong and nothing downstream can fix it.

The Conversation Nobody Wants to Have

Sometimes we don’t know how hard a recruit will be until we’re in it. You build your screener, you set your quotas, and then reality hits — the incidence rate is lower than expected, or a qualifying criterion eliminates more people than anyone predicted. Or you discover that the people who do qualify are all at the same industry conference that week and nobody’s answering emails.

That’s when we have the uncomfortable timeline conversation. Not because anyone did anything wrong, but because research involves people, and people don’t always cooperate with project plans. We’d rather be honest about that upfront than deliver a half-baked sample on deadline. A study with the right fifteen people will always beat a study with fifty wrong ones.

That cement project? We got it done. It took creative recruiting, adjusted timelines, and a few favors called in. The data was solid because we didn’t cut corners on who we talked to.

So if you’re planning research and someone asks “who’s your audience?” — take that question seriously. The answer will shape more of your project than you’d expect.