Humans set the benchmark
Before scoring Candor, we measure how much humans agree with each other on the same task. Synthetic results are reported against that number, not against 100%.
Humans don’t fully agree with each other, so synthetic participants can’t be scored against perfection. We score Candor against how much humans agree among themselves, and have the transcripts scored blind.
Where participants come from
Before you can trust what a synthetic participant says, you need to know where they came from.
Before anything is generated, you set who the audience is and what to include or exclude, the same way you’d brief a recruiter.
It searches public web sources about the audience. You can also upload your own material, such as past research, survey results and interview transcripts.
Candor turns that evidence into 15 to 24 participants who differ from each other, not copies of one typical person. Each has their own background, role, habits and attitudes.
Each profile detail is tagged Grounded (taken directly from a source), Derived (reasoned from sources) or Assumed (filled in where the sources ran out).
Latest result
We ran a concept test with 12 human nurses, then ran the same test with 16 of Candor’s synthetic nurses. Both groups sorted the same six concepts and explained their choices.
Two groups of human nurses don’t give identical answers. That agreement is the benchmark. Here’s how close synthetic nurses came to it.
One study, one audience (US nurses), October 2026. We’re running more, and the number will change as we do. Read the full write-up.
How we test
Before scoring Candor, we measure how much humans agree with each other on the same task. Synthetic results are reported against that number, not against 100%.
Two AI models from different companies score transcripts without being told which are synthetic. We report how often they agree.
Who runs the tests
Head of Research. Social and personality psychologist, trained in experimental design and psychometrics. Designs and runs every test on this page.
Chief Product Officer. Co-author of Lean Analytics. Built by Highline Beta, a venture studio.
In our latest test, Candor's synthetic nurses reached 87.4% of the agreement human nurses reached with each other. Twelve human nurses and 16 synthetic nurses sorted the same six concepts and explained their choices. That is one study with one audience (US nurses) in October 2026. We're running more, and the number will change as we do.
Because real people don't fully agree with each other, so 100% is not a fair target for anyone. Before scoring Candor, we measure how much humans agree with each other on the same task. Synthetic results are then reported against that human benchmark. A score near the benchmark means synthetic participants agree with people about as well as people agree with each other.
The scoring is blind. Two AI models from different companies score the transcripts without being told which ones are synthetic, and we report how often the two agree. Humans also set the benchmark: we measure how much people agree with each other first, and report synthetic results against that number.
They are built from evidence. You define the audience. Candor searches public web sources about it, and you can upload your own research, surveys and interview transcripts. That evidence becomes 15 to 24 participants who differ from each other. Every profile detail is labeled Grounded (taken directly from a source), Derived (reasoned from sources) or Assumed (filled in where the sources ran out).
Yes. Take a study you've already run with real people and run the same study in Candor. Compare the two to see whether Candor tracks your results, what it misses, and what it surfaces that you didn't expect. You can start with a free project.
Take a study you've already run with people and run it in Candor. See whether it tracks, what it misses, and what it surfaces that you didn't expect.