Reading a problem validation report

Validation is the type that forces an answer, so its report is organised around the hypotheses you wrote. This article covers the parts specific to validation. For the sections every report shares, plus confidence levels, exports, and the thin-run banner, start with Reading a synthesis report.

The five verdicts

Every hypothesis you tested comes back with one of five verdicts. Three of them are results. Two of them are telling you something about the study rather than the hypothesis.

  • Validated. The evidence supports it.
  • Partially validated. It held for some of the audience and not others. Check the per-archetype read in the appendix before you act, because this often means the hypothesis is true of a segment rather than the market.
  • Not supported. The evidence goes against it. This is a result, and a cheap one.
  • Not applicable to audience. The hypothesis is about something this audience doesn’t do. That is a targeting signal, not a hypothesis signal.
  • Inconclusive. The interviews didn’t produce enough consistent evidence either way. Usually a hypothesis that was too vague to test.

The lead adapts to the result

The headline at the top of What Matters Most is not a fixed template. When the run escalates something, or when the verdicts came back largely inconclusive, the lead changes to say so rather than reporting a tidy result over an untidy study. If the top of your report opens by telling you the study itself is the problem, believe it and fix the hypotheses before reading further.

A map, not a verdict

Under the learnings you’ll find a short note framing the report as a map rather than a ruling. It is worth taking literally. Synthetic personas can tell you a problem statement is coherent, widely recognised, and consistent with how this audience behaves. They cannot tell you someone will pay to have it solved. Use validated hypotheses to decide what to build next and what to take to real customers, not as the final word.

What’s in Findings

  • An at-a-glance table. Every problem in one collapsible view, for when you want the shape before the detail.
  • Tested hypotheses. One card per hypothesis with its verdict and a count in the form “X of N personas”, each collapsing its own detail and evidence. The count is the thing to read alongside the verdict: Validated at 3 of 12 is a different fact from Validated at 10 of 12.
  • Problems that surfaced on their own. Pain raised without being asked, outside your hypothesis set. See the question below on why this is often the most valuable section in the report.

In the appendix

Two things sit in the evidence appendix rather than the findings.

  • Sort distribution, present only when the run used a card sort. That happens automatically once a guide carries five or more hypotheses.
  • How each archetype read the hypotheses. The place to go whenever a verdict came back Partially validated, because it usually resolves into one archetype agreeing and another not.

Where to go next

Common questions

Usually nothing broke. Inconclusive means the interviews didn't produce enough consistent evidence either way, and the most common cause is a hypothesis that was too vague to test. "Users struggle with onboarding" can't return a clean verdict because every persona will read it differently. "New users abandon setup because they can't tell which integration to connect first" can. The other common cause is that the hypothesis was outside what this audience does at all, which returns Not applicable to audience rather than Inconclusive. Sharpen the statement and re-run rather than reading Inconclusive as a weak no.

Because it's often where the real finding is. Problems that surfaced on their own collects pain personas raised without being asked, outside your hypothesis set. A validation study is by design narrow: you go in with statements and check them. That narrowness is what makes the verdicts trustworthy, and it's also what would make you miss the thing you didn't think to write down. If a problem in this section reached more personas than any of your tested hypotheses, that's your next study.

It appears only when the run used a card sort, which happens automatically once you give the guide five or more hypotheses. Verdicts tell you what held. The sort distribution tells you how personas ranked the hypotheses against each other before any deep-diving happened, which is a different signal: a hypothesis can be validated and still have sorted near the bottom for everyone, meaning it's real but not urgent. Read them together when you're deciding sequence rather than just what's true.

More FAQs →

Candor is in development.

Be the first to know when it launches.

No spam. Just a note when Candor is ready. Powered by Highline Beta.