There's a strong case against synthetic research going around, and if you're weighing it as a method, you should read it. The bluntest version ran under a headline that doesn't hedge: synthetic users don't work. It's built on a systematic review of 182 studies spanning psychology, healthcare, marketing, and six other fields.
I think a lot of it is right.
That's not the usual move for someone who builds a synthetic research platform. But the criticism is good, and pretending otherwise would be dishonest. The failures it documents are real. They're the failures you'd expect if you asked a general-purpose chatbot to play your customer and trusted whatever came back.
Where I land differently isn't on the failures. It's on what they add up to, and on what synthetic research is actually for.
What the critics actually found
The strongest critique isn't "AI is bad." It's specific, and it lands in four places: synthetic respondents don't reason like people, they flatten into stereotypes, they hallucinate, and worst of all, they look convincing while doing it.
The flattening is worth digging into. Ask an ungrounded model to name an artsy video game and it overwhelmingly picks the same over-discussed indie title, because that's what the internet talked about most. Real people answer all over the map. The review calls low diversity "perhaps the most universal and ubiquitous bias" in synthetic data. Models give you the averaged residue of everyone's opinion, not the spread of actual human difference.
This isn't a fringe worry, either. When User Interviews surveyed 150 researchers, 47% described themselves as skeptical and wanting more evidence before they'd trust it. Their top three concerns: the quality of the insights (88%), stakeholders over-trusting AI output (79%), and bias getting amplified across underrepresented groups (77%). These are reasonable people naming reasonable problems.
The believability trap
The most dangerous failure isn't that synthetic output is bad. It's that bad synthetic output looks good.
The review has a phrase for this: "misleading believability." The text comes back clean, structured, elaborate, full of complete sentences and confident opinions. It fills the template. It feels like data. Experts in one study couldn't tell synthetic personas from human-generated ones on a first read.
And underneath, it's shallow. Stereotyped. Occasionally fabricated. The believability is the trap, because it lends false credibility to conclusions that don't deserve it. A confident fiction is worse than an obvious one. The obvious one gets caught. The confident one gets put in a deck and shipped.
This is the real reason "the AI said so" should scare you. Not because the AI is dumb. Because it's fluent, and fluency is easy to mistake for truth.
What we do about flattening
Flattening is the failure I care most about, because it's the one that quietly ruins research. If every persona drifts to the same agreeable, middle-of-the-road answer, you've interviewed one person wearing eight name tags.
So Candor doesn't hand a study a single personality and clone it. It clusters the audience into distinct archetypes from the evidence, samples each persona's personality and cognitive biases from calibrated ranges, and enforces real separation between personas in the same archetype so siblings don't collapse into near-twins. B2B and B2C run on different models, because the two reason differently. The mechanics are in how archetype clustering works and the OCEAN model in synthetic personas.
I won't tell you this eliminates flattening. Nothing does today. But there's a real difference between a tool that samples genuine variance on purpose and one that lets the model fall back to its blandest, most-probable answer. The critics were describing the second kind. We built against it.
Grounding helps. It doesn't make synthetic human.
The review doesn't claim nothing helps. It says grounding in real, context-specific evidence, better persona design, and cognitive modeling all improve the output. It also says the improvements stay modest, and that none of them turn a synthetic respondent into a stand-in for a real one. I think that's a fair reading of the evidence.
So I'm not going to tell you grounding closes the gap. It doesn't. A grounded synthetic persona still isn't a person, and anyone selling you that is selling the exact thing this review warns about.
What grounding does is narrower and more useful. It's the line between synthetic research that's worth running and synthetic research that's hollow. In one project, Bain reports backtesting grounded synthetic customers against a prior conjoint study: they built the respondents from historical first-party data, deliberately held the original study out of the inputs, and say the synthetic output matched about 90% of the key outcomes. It's a consultancy describing its own client work, not an independent trial, so take the exact number with that in mind. The part that holds regardless is the design. Holding the answer out and then checking the work is the right way to test this, and their broader takeaway lines up with everyone else's: the data you ground the model in matters more than which model you pick.
That's not "synthetic equals human." It's "grounded synthetic can do real work on the right questions." And the right questions are where the debate should actually be.
So what is synthetic research actually good for?
Not replacement. Augmentation. Every credible source lands in the same spot, including the critical ones.
The pattern that works is hybrid. Synthetic goes first to map the terrain and kill the weak ideas. Human research goes next to explain why the patterns exist and to make the high-stakes call. As one industry writeup put it, synthetic research "is not a shortcut for skipping human research. It's a tool for investing human research time more deliberately." Bain's guidance to teams starting out is two words: "augment rather than replace."
Even the 182-study review leaves room for this. It calls synthetic participants "heuristic-like" and points to supplemental uses: cold-start problems, piloting instruments before you bring in humans, interrogating hypotheses, low-stakes questions, and hard-to-reach groups. That's augmentation. It's a narrower claim than the marketing, and it's the honest one.
So the honest scorecard looks like this:
-
Synthetic is strong at relative comparison. Which of these five concepts is strongest, which positioning resonates, where a value prop is weakest. Ranking questions, not magnitude questions.
-
Synthetic is strong at breadth and early screening. Surfacing and stress-testing assumptions across a whole spec, including the low-stakes ones you'd never put in front of a panel.
-
Synthetic is weak at statistically-bounded numbers. It can show you direction: where price acceptance breaks, which way a segment leans, which option ranks ahead of another. It can't give you "23% of buyers will switch at this price, plus or minus 3 at 95% confidence." That interval needs real-respondent N. Read synthetic price and preference output as direction and ranking, not as the final number you set the price by. (When Candor runs price testing, it deliberately reports a directional range and refuses to emit a single point estimate, for exactly this reason.)
-
Synthetic is weak at anything requiring real lived experience or in-product behavior. It reasons about behavior. It doesn't perform it. Usability on a live prototype stays with real users.
Know which question you're asking, and synthetic earns its place. Forget which question you're asking, and you walk straight into the believability trap. This is the core of what synthetic user research actually is, and the conditions where concept testing with synthetic users works get specific about the boundary.
How to tell rigorous synthetic from the hollow kind
Ask the vendor four questions. The answers sort the category fast.
Where does the evidence come from? Specific sources you can name, or general AI training data? Rigorous synthetic starts by retrieving real evidence about your audience before it generates anything. We built Candor this way on purpose, with every persona attribute tagged for where it came from. The full pipeline is in how evidence grounding works.
How is personality calibrated? Named, validated frameworks, or vibes? Flat agreeableness is what produces the stereotyping the critics caught. Sampling personality and bias from calibrated ranges is how you get the human spread back, which is the whole job of archetype clustering.
How is consistency enforced? A real validation mechanism, or does the persona contradict itself by question seven?
And how does the tool handle the documented tendency of these models to just agree with you? If the answer is silence, you've found an AI persona generator wearing a research costume.
If you want the longer version of this rubric, the evaluator's framework for synthetic research tools breaks the whole category down without the marketing gloss.
What it comes down to
The skeptics did the field a favor. They drew a hard line between synthetic research that's grounded, calibrated, and validated, and synthetic theater that generates confident-sounding fiction. Most of what they attacked deserved it.
One honest note on that 182-study review, because you should weigh the source. It's a preprint, not yet peer-reviewed, and two of its three authors work for a human-based UX research platform, the kind of business synthetic users would compete with. So read their framing with that in mind. They have a reason to be hard on synthetic. What keeps the findings credible anyway is that it's a review of 182 independent studies, not their own experiments, and the same failure modes turn up across all of them. Even discounting for the source, the core result holds: ungrounded synthetic, used as a substitute for people, doesn't work. I agree with that. I just don't think substitution was ever the point.
Grounded and used for the questions it fits, synthetic research holds up. Ungrounded and used as a replacement for talking to people, it's exactly what the critics say it is. That's the whole distinction, and most of this debate is people talking past each other because nobody bothered to name it.
So if you're evaluating synthetic research, the question that sorts it is simple: is the output grounded in real evidence you can trace, or is it a chatbot imagining a customer for you? See how Candor works for our answer, and the use cases where the grounded version earns its keep. What's the failure that made you skeptical of synthetic research in the first place?