Synthetic participants run on language models that were tuned to agree with people. Left alone, they tell you what you want to hear. Candor counters that on both sides of the interview: the participant answering and the interviewer asking.
This post explains what each safeguard does and where it stops.
Why synthetic participants agree too easily
Contrary Research's report AI Polling: How Synthetic Surveys Could Predict Real Behavior, by Claire Burch, collects the evidence. It cites Perez and colleagues, who showed that training models on human preferences makes them more sycophantic, meaning more likely to flatter and agree. Pollster John Hagner reports that synthetic respondents won't get as negative as real people do. Models also rarely say "I don't know."
One pollster in the report puts the risk simply: you end up asking the machine to confirm what you already believe.
We agree with that criticism. We covered the rest of the report in synthetic interviews are not synthetic polls, so this post stays on one problem: people-pleasing.
That problem has two sources. The participant can be too agreeable, and the interviewer can lead the participant toward the answer it expects. A synthetic interview has a model on both sides, so Candor needs safeguards for both.
How Candor keeps participants from agreeing by default
Each participant has a set skepticism level. There are four levels: accepting, questioning, wary and challenging. Candor assigns each participant a level based on their personality traits (mainly how agreeable and how anxious they are) and on how strongly they defer to authority. The level is set when the participant is built, before any interview starts.
Each level carries a target for how often the participant pushes back. A challenging participant probes about one in three claims before engaging with them, and sometimes questions the question itself. A wary participant raises a worry first, such as what it would cost them if the idea failed. A questioning participant speaks up when something conflicts with their own experience, and an accepting participant rarely challenges a premise.
We keep accepting participants on purpose, because some real people are accepting. But going along with a framing isn't the same as liking an idea, and the participant is told so. Whether they'd use it, pay for it or trust it still comes from their own situation. The most skeptical participants get the rule in plain words: being pleasant does not mean being agreeable.
Participants have their own views to defend. A participant can only disagree with you if they have views of their own. Candor builds each participant from evidence about your audience, gathered before anyone is generated, and your own uploaded research ranks above anything from the web.
Each detail describing who a participant is, how they behave and what they believe is tagged Grounded (something you can check), Derived (follows from the evidence) or Assumed (Candor filled it in and says so). How evidence grounding works covers the full pipeline.
Participants are allowed to say "I don't know." They're told to say "I'm not sure" when a question falls outside their experience, rather than invent a detailed answer. They can't name real brands or products, or give exact figures for what they spend, unless those are already in their memory. Hedging doesn't get around the rule: "maybe it's QuickBooks?" still counts as naming a brand.
Each answer also records any topic the participant felt was outside their experience, and the transcript shows it.
A second model checks answers for contradictions. Each answer is reviewed against what the participant has already said, believed and decided. If it clearly contradicts that record, it's sent back and rewritten, up to twice. A softer shift, such as a sudden rise in enthusiasm, is flagged but still delivered, because real people's moods change too.
This catches a participant who reverses a stated position because a question pushed them. It won't catch one who warms up gradually over a long conversation.
How Candor keeps the interviewer from leading
In an automated interview, Candor writes and asks the questions. A leading interviewer can pull agreement out of even a skeptical participant, so the interviewer gets the same scrutiny as the participant.
The unprompted answer comes first. In problem validation, the interviewer asks about the participant's own problems before naming yours. In a concept test, it captures the first reaction before explaining the concept. In a price test, it asks for the participant's own price guess before mentioning a number.
The order matters because it can't be undone. Once a participant hears your framing, you can't get the unprompted answer back.
Leading phrasing is banned. The interviewer never asks "what do you like about this?" or "what did you love about that?", because both assume a positive answer. "Wouldn't you agree" and "most people tell us" are banned too, along with forced choices between two options the interviewer made up. The interviewer also doesn't open with praise like "that's really helpful," because praise signals which answers you want.
A separate model reviews the interviewer's questions. Before a question the interviewer wrote goes out, a second model checks it for leading patterns. It looks for signaling phrases, social proof, forced choices, assumptions the participant never stated, and a concept or price shown too early. Serious problems send the question back to be rewritten, and the rewrite is checked again. Milder problems, like two questions stacked into one, are logged.
Some sections need a question shape that would count as leading anywhere else. Problem validation has to put your hypothesis in front of the participant at some point. Those sections have written exceptions tied to their method, and the check still flags any question that goes further than the method requires.
The guide asks for disagreement directly. Problem validation interviews end with three fixed questions that invite pushback. They ask whether anything about how the problems were described doesn't fit the participant's reality, what you'd miss by only talking to people like them, and who would see it completely differently.
Concept tests include an objection probe the interviewer can't skip. A participant who hasn't raised an objection may simply not have been asked.
In live interviews you write the questions yourself, so the interviewer check doesn't run on them. The participant safeguards still apply.
How we test these safeguards
We run separate validation tests, outside customer studies, to check whether the safeguards hold.
The first asks the same participant the same question twice. One version is neutral, and the other opens by saying most people loved the idea. The test measures how far the answer moves. A participant with a real position shouldn't move much.
The second checks whether participants push back about as often as their skepticism level says they should. It used to run on every interview turn. We retired that version in August, so it now runs only in test campaigns, not during your interviews.
The third measures how different participants' answers to the same question are from each other. If every participant gives the same agreeable answer, this test shows it. Low variety is one of the failures synthetic research skeptics get right.
Where the safeguards stop
Social desirability bias has no dedicated safeguard. This is the habit of answering the way a "good person" would, such as overstating how often you exercise or read the fine print. Real participants do it too, which is why good interviewers ask about past behavior instead of intentions. Candor's guides follow that practice, but nothing in the participant is built specifically to counter the bias, and the models' training pushes toward the virtuous answer.
Synthetic participants are tamer than real ones. The underlying models are trained to stay civil. A challenging participant will tell you plainly that they doubt your idea. They're unlikely to be rude about it, and some of your real customers might be.
Real conversations still matter. Synthetic interviews help you narrow your options and find objections you hadn't considered. Take what survives to real people, especially before a decision you'll be held to. If an entire group of synthetic participants agrees with everything, look closer before you trust it.
Where to go next
If you're weighing synthetic user research, start with synthetic interviews are not synthetic polls for the wider set of criticisms. How to spot convincing nonsense covers what to watch for in the output, and how Candor works walks through the full pipeline.