Ask AI for quotes from a transcript, then check every one. How many did people actually say?
What
Give AI a transcript, ask it for supporting quotes, then check each quote word for word against the original.
Why
AI summaries look trustworthy because they come with quotes. But a quote that's been tidied, merged or invented is no longer evidence. If it reaches a stakeholder, it can steer a decision on something nobody said. This experiment shows you how much checking your tool really needs.
How
1Pick one anonymised transcript you know well.
2Ask:
Identify the three main themes in this interview. For each theme, give two direct quotes from the participant, word for word, exactly as written in the transcript.
3Search the transcript for each quote (Ctrl+F or Cmd+F).
4Mark each one: exact, tidied (meaning kept, wording changed), merged (two moments joined together) or not there.
What to watch for
- ○How many of the six quotes were exact?
- ○Did tidied quotes change the meaning, for example by making someone sound more certain or more negative?
- ○Were the themes still right, even where the quotes weren't?
Tried it?
What was your score out of six?
Tell us what happened, good or bad. Other members are trying it too.
Discuss in our LinkedIn Group