What Thematic Analysis Looks Like in a Real Team
The textbook version of thematic analysis runs in six clean phases: get familiar with the data, generate initial codes, search for themes, review them, define and name them, then write up. Actual teams don't work that way. There are 14 transcripts, a readout on Thursday, three people coding in a shared board, and a product lead who already has a hunch about what the answer is.
Most teams get from transcripts to themes one way or another. The harder question is whether those themes would survive someone pulling up the transcripts and checking them. In our experience that check fails more often than teams expect, and it tends to fail in the same place. The coding is usually fine. What goes wrong is the gap that opens between a theme's label and the evidence filed under it.
We call this Theme Label Drift. A theme's name gets revised more broadly each time the team touches it, while the excerpts underneath stay the same. By the readout, the label is making a claim the excerpts never made.
How Theme Label Drift Happens
Here's a typical case. In a study of onboarding for a B2B analytics tool, one researcher codes four excerpts as "confused by permissions screen." They're specific: participants didn't understand the difference between viewer and editor roles.
At the first clustering session, those four codes sit next to two others: "didn't know where to invite teammates" and "unsure who owns the workspace." Someone suggests a parent theme: "Collaboration setup is confusing." That's reasonable, and it still sits close to the data.
At the second review, the theme gets merged with a cluster about Slack integration friction. The new name is "Teams struggle to adopt collaboratively."
By the readout deck it's become "The product doesn't support how teams actually work," and it's presented as the study's headline finding, the kind that justifies a quarter of roadmap work.
No step in that chain was dishonest. Each rename was a small, defensible generalisation of the one before. But nobody went back to the original seven excerpts and asked whether they support the final wording. They don't. They support "the role and invite model on the permissions screen is unclear." That's a two-week design fix. The headline implies a strategic rethink.
The mechanism
Three forces cause the drift.
1. Labels get edited more often than evidence. Renaming a sticky note takes seconds. Re-reading 30 excerpts takes an hour. When review time is tight, the cheap operation gets done over and over and the expensive one doesn't get done at all. Each review pass moves the label and leaves the evidence where it was.
2. Merging pushes toward abstraction. When two themes combine, the new label has to cover both, so it's always broader than either. There's no step in the process that pushes a label back toward something more specific. After three merges, a theme's name has usually become a claim about the whole product.
3. The audience rewards scope. Stakeholders pay attention to claims about strategy and ignore claims about a single screen. Researchers know this, even if they wouldn't say it out loud, and the final rename is often written for the audience rather than for the data. It's the same pull that gives a single memorable quote too much weight, which we covered in the verbatim overweighting effect. Here it acts on the label rather than the quote.
Engineers have a close parallel. When a retrieval system keeps its old vectors but changes the model that interprets them, the answers degrade with no visible error. Bigyan's write-up on embedding drift in RAG systems describes this exact pattern: what's stored stays the same, the layer that interprets it moves, and every dashboard stays green. Theme labels are the interpretive layer of a qualitative study, and they drift in the same quiet way.
Why It Matters: The Decisions That Go Wrong
Theme Label Drift doesn't produce false findings. It produces findings that are true at the wrong scale, which is arguably worse because they hold up to casual scrutiny. If someone challenges the theme, the researcher can point to seven real excerpts. The excerpts just support a smaller claim than the one the team is acting on.
Three kinds of decision go wrong as a result:
- Scope inflation. A usability fix gets treated as a strategic problem, and the team funds a platform rework when a redesigned modal would have done the job.
- Prevalence inflation. A broad label draws in more excerpts over time, so the theme looks common. "Seven of twelve participants" sounds strong until you notice that four of those seven were talking about something different. This also feeds the problem in the saturation reporting gap: a label broad enough to absorb any new interview will always look saturated.
- Lost specificity. The actionable detail, which role label confused people, is removed by the renaming. Even when the theme is right, the team building the fix can no longer see what to fix.
AI-assisted synthesis makes this worse if nobody checks it. A model asked to "summarise the key themes" generalises by default, and it will produce the broad label in one step instead of three. We've written about the related failure in consensus manufacturing in AI summaries. The fix is the same in both cases: tie every claim to the evidence under it and make the tie visible.
The Counter-Practice: Evidence-Locked Themes
The goal isn't to ban abstraction. Themes are supposed to generalise beyond individual codes. The goal is to make every generalisation earn its place by being checked against the evidence again. Here's the workflow we recommend.
1. Write themes as claims, not topics
"Collaboration" is a topic. "Admins can't predict what an invited teammate will be able to see" is a claim. A topic label can drift without anyone noticing, because it doesn't commit to anything. A claim can be tested against an excerpt, and that's the whole point. Every theme should be a full sentence with a subject and a verb.
2. Version the label, not just the board
Every time a theme is renamed, keep the previous name alongside it. By the readout you should be able to show the chain: "confused by permissions screen" to "collaboration setup is confusing" to the final wording. If a stakeholder sees that chain, they can judge for themselves whether each step was justified. If you'd be uncomfortable showing it, that tells you something.
3. Run the re-read test at every merge
When two themes merge, pick three excerpts at random from the combined set and read them aloud against the new label. Ask one question: would this participant recognise themselves in this sentence? If the answer is "sort of" for any of them, the merge has gone too far. Split it, or narrow the label.
This takes about five minutes per merge. On a typical study with eight to twelve merges, that's under an hour, and it's the most valuable hour of the analysis.
4. Separate the evidence claim from the implication
Split the readout into two layers and label them clearly:
- Finding: "Six of twelve admins misread the viewer/editor distinction on the permissions screen." This is what the excerpts support.
- Implication: "This may reflect a broader mismatch between our role model and how teams delegate." This is the researcher's interpretation.
Both belong in the readout. Theme Label Drift is what happens when the implication gets presented as if it were the finding. Keeping them visibly separate lets stakeholders see how much of the claim rests on evidence and how much rests on judgment.
5. Count only excerpts that match the final label
Prevalence counts should be recalculated after the last rename, not carried over from the clustering session. Re-check every excerpt against the final wording and count only the ones that clearly fit. The number will usually drop. That drop is useful information.
6. Have someone who wasn't in the clustering sessions audit the headline themes
People who were in the clustering sessions have absorbed the renaming history and read the drift as natural progression. A colleague who wasn't there sees only the final label and the raw excerpts, which is exactly the view a sceptical stakeholder will have. Give them the top three themes and the evidence and ask them to restate each theme in their own words. If their version is narrower than yours, trust theirs.
A Worked Example
A 20-person research consultancy running a patient-experience study for a mid-size healthcare network had a theme in its draft readout called "Patients distrust the digital front door." It was based on nine excerpts across fifteen interviews.
An audit with the re-read test found:
- Four excerpts were about appointment reminders arriving from an unfamiliar phone number, which is a trust issue but a narrow and fixable one.
- Three were about not knowing whether a portal message would reach a clinician or an administrator. That's about routing uncertainty, not distrust.
- Two actually expressed broad scepticism about digital channels.
The revised readout had three themes. The reminder issue went to the ops team as a fix that could be done within a week. Routing uncertainty became a design brief for the messaging interface. The broad distrust finding was kept, marked as two participants, and flagged for follow-up research. The client would have funded a trust-and-brand initiative from the original headline. They got three decisions of the right size instead.
This is the kind of structure we describe in the evidence density test for research reports: each claim sits directly next to the excerpts that support it, and the reader can see how much evidence each claim has.
Where Tooling Helps and Where It Doesn't
Software can't make the interpretive calls for you. It can make them visible and cheap to check. Qualz.ai links every theme back to its source excerpts and timestamps, so running the re-read test means clicking through rather than searching transcripts. When themes merge, the underlying evidence stays attached and can be inspected, and prevalence counts reflect the excerpts actually linked, not a number someone typed in during the first session. The platform's AI-assisted coding proposes themes as claims tied to specific evidence, and a researcher still decides whether each generalisation holds.
That split is how it should work. The machine handles the bookkeeping that tired teams skip, and the researcher does the judgment that nobody should hand off.
Practical Takeaways
- Write every theme as a full-sentence claim with a subject and a verb, never as a one-word topic.
- Keep a rename history for each theme and be ready to show the chain from first code to final label.
- At every merge, read three random excerpts against the new label. If any participant wouldn't recognise themselves in it, split or narrow the theme.
- Split findings from implications in the readout, and label each layer explicitly.
- Recount prevalence against the final label, not the clustering-session tally, and report the lower number.
- Have someone who wasn't in the clustering sessions audit your top three themes before any readout that will drive roadmap or budget decisions.
- Treat a narrower theme as a better one when it comes with a specific, fundable fix.
If you'd like to see how evidence-locked theme development works on your own transcripts, book a Qualz.ai information session and bring a recent study. We'll run the re-read test on its headline theme together.
