Judge reliability
How reliable a result is and how to check the evidence behind it.
A result is only as good as the answers behind it. The report shows you in several places what a statement rests on and where you should be careful.
Markers in the report
| Marker | Where | What it means |
|---|---|---|
| Dynamic Personas | Report head | The answers come from Dynamic Personas. Its hint reads Simulated answers. The results show tendencies. |
| Premium Twins | Report head | The answers come from Premium Twins of real people who were interviewed in depth beforehand. |
| n = 50 | Report head, charts, KPIs | How many answers stand behind the value. Answers that failed for technical reasons are not counted. |
| No answer (technical): 3 | Questions, KPIs | How many answers failed for technical reasons despite retries. |
| Not interpretable | Questions, KPIs | More than 20 % of the answers failed. There is only the distribution, with no reading, no themes and no evidence. |
| Machine translated | Evidence | The quote was translated into your language automatically. |
| Quote unavailable | Evidence | A piece of evidence could not be loaded. |
Dynamic Personas or Premium Twins
Read the report head first. It tells you which kind of panel answered:
- Dynamic Personas are AI respondents generated from population statistics. Their results show tendencies: which idea lands better, which reasons come up, where there is resistance. Take exact percentages as a direction, not as a market share.
- Premium Twins mirror real people who were interviewed in depth beforehand. The answers are closer to real individuals. Premium Twins are pseudonymous: respondents appear only with a number and a few attributes, never with a name.
Also watch n. With 20 respondents, a single answer moves a share by 5 percentage points.
Open the evidence
Themes of open questions, charts and the Why of a recommendation carry a quote icon. Hover over it to see the number of pieces of evidence, for example Evidence (3).
For themes of open questions it reads Selected evidence (11). The theme's count, for example 14 / 20, covers every answer that belongs to the theme. The evidence is a selection of verified quotes from those answers, so there are often fewer. An answer can count toward a theme without appearing in its evidence: the evidence is the examples the analysis cites, not every matching answer.
- Click the quote icon. The Evidence panel opens on the right.
- Read the quotes. Under each quote are attributes of the respondent, such as gender, age group and region.
- For evidence from scale or choice questions, you see the question's distribution instead of a quote, for scales with the Mean = 4.2.
- Close the panel with the X (Close Evidence).
The summary has no evidence of its own. What it says rests on the recommendations and questions below it.
Original and translation
The survey currently runs in German. When you read the report in English and a translation exists for a quote, it first shows the translation with the Machine translated marker. Without one, the quote appears in German. Show original shows the German wording, Show translation switches back. Quote the original when the exact wording matters. More in Languages.
Failed answers
When an answer fails for technical reasons, it is not guessed or filled in. It appears as No answer (technical) next to the question and is not part of n. Your team gets it refunded. If the share for a question exceeds 20 %, it is marked Not interpretable: the distribution of the answers that did come in stays visible, but the summary and recommendations do not rely on it. More in Problems & refunds.
How to approach a report
- Check the report head: panel type, panel version and n.
- Separate measurement from interpretation: charts, shares and theme counts are measured. Readings, the summary and recommendations are conclusions the analysis draws from them.
- Read the evidence: before you pass on a recommendation, open its evidence and check that the quotes support it.
- Argue with quotes: when you present a finding, use the quote from the evidence, not the summary.