Chapter 9: Analysis & Synthesis

The Goal of Analysis

The purpose of analysis is to produce decisions, not reports ^{just-enough prr}. A beautifully formatted research deck that sits on a shared drive has produced zero value. A rough synthesis session that changes what the team builds next has produced immense value.

This distinction matters because it shapes how you allocate your time. Teams that treat analysis as a deliverable spend weeks polishing presentations. Teams that treat analysis as a decision-making activity spend hours in a room with sticky notes and walk out with a clear next step.

The question that should guide every analysis session is: "What should we do differently based on what we learned?" If you cannot answer that question, either the research was unfocused, the analysis is incomplete, or the finding is not actionable — in which case, note it and move on.

Affinity Diagrams: Bottom-Up Qualitative Clustering

The affinity diagram is the workhorse of qualitative analysis ^{just-enough prr}. It is a bottom-up clustering technique that lets patterns emerge from data rather than imposing categories on it.

The process:

1. Externalize: Write each discrete observation, quote, or data point on its own sticky note (or digital equivalent). One insight per note. Include just enough context to understand the note on its own — participant identifier, the situation, and the observation.

2. Spread: Place all notes on a wall or large surface where the entire team can see them. Do not organize yet.

3. Cluster silently: Team members move notes into groups based on similarity. No talking during this phase — it prevents the loudest voice from imposing premature structure. If two people disagree about where a note belongs, duplicate it.

4. Name the clusters: Once groups have stabilized, the team discusses each cluster and gives it a descriptive label. The label should capture the insight, not just the topic. "Users are confused by pricing" is better than "Pricing" as a cluster name.

5. Identify themes: Look for relationships between clusters. Which clusters connect? Which are in tension? Which are surprising?

The power of the affinity diagram is that it prevents top-down categorization. When you start with categories ("usability issues," "feature requests," "positive feedback"), you force data into predetermined buckets and miss the patterns that do not fit. When you start with individual data points and let them attract each other, the categories that emerge are grounded in reality.

See Affinity Diagrams for a detailed facilitation guide.

Collaborative Analysis

Analysis improves dramatically when it is done by the team, not delegated to one person ^{prr sprint}. Three reasons:

Shared evidence creates alignment. When everyone on the team has seen the same data and participated in the same synthesis, decisions do not require lengthy persuasion. The team converges because they reached the same conclusions through the same process.

Multiple perspectives catch blind spots. An engineer notices technical constraints in user workarounds. A designer notices interaction patterns. A product manager notices business model implications. A single analyst, no matter how skilled, cannot hold all these lenses simultaneously.

Analysis builds empathy. There is no substitute for watching five customers struggle with the same step, or hearing three different people describe the same frustration in their own words. When the team does the analysis work together, empathy for the customer is distributed across the team, not concentrated in the researcher's head.

The practical requirement: schedule analysis sessions within 24 hours of data collection, while memories are fresh. A one-hour team synthesis session the afternoon of the interviews is worth more than a polished analysis document delivered two weeks later.

The Sprint Analysis Method

The design sprint uses a structured, real-time analysis approach that works well beyond the sprint context ^sprint.

The grid: Create a table on a whiteboard with one column per customer interviewed and one row per major scene or task in the prototype. As each interview proceeds, team members watching from the observation room write notes on sticky notes and place them in the appropriate cell. Use different colors for positive reactions, negative reactions, and neutral observations.

Pattern identification: After all five interviews, the team reviews the grid column by column (looking at each customer's full experience) and then row by row (looking at each scene across all customers). A pattern that appears in three or more columns (three of five customers) is a strong signal — reliable enough to act on.

The +/- summary: For each row, mark whether the pattern was positive (+), negative (-), or mixed (~). This creates a quick visual summary of where the prototype succeeded and where it failed. The team can then prioritize: fix the negatives, preserve the positives, investigate the mixed signals.

This method works because it produces a shared, visual artifact that the entire team contributed to. There is no "findings presentation" because everyone was present for the findings.

The Interview Snapshot

Teresa Torres introduces the interview snapshot as a lightweight analysis artifact that the product trio creates after every customer interview ^cdh. It captures:

• Key quotes (verbatim, in the customer's own words)
• Opportunities identified (unmet needs, pain points, desires)
• Insights (things the team learned that change their understanding)
• Quick sketch of the opportunity or experience being described

The snapshot is deliberately brief — one page, created in fifteen minutes immediately after the interview. It is not a transcript or a detailed report. It is a synthesis tool that forces the team to extract and commit to the key takeaways while the conversation is still vivid.

Over time, snapshots accumulate into a library that the team can review to identify recurring patterns. When the same opportunity appears in five snapshots from different customers, it has earned serious attention.

See Interview Snapshots for a template and usage guide.

Three Modes of Analysis

Lombardo and Bilgen describe three complementary modes of analysis ^prr:

Playing (Collaborative and Physical)

Analysis as a team activity with physical materials — sticky notes on walls, sketches on whiteboards, role-playing scenarios. This mode leverages spatial reasoning and group energy. It is best for generative synthesis: finding new patterns, making unexpected connections, and building shared understanding.

Playing works well early in the analysis process when you are still figuring out what the data means. The physicality helps — moving sticky notes is cognitively different from reading a document, and the difference produces different insights.

Making (Prototypes and Artifacts)

Analysis through creation. Build a journey map, sketch an experience diagram, or create a prototype that embodies what you learned. The act of making forces clarity because you cannot build something vague. Every element in the artifact must be specific, which exposes gaps in your understanding.

Making works well as a bridge between analysis and action. Instead of writing a report that describes what you learned, build an artifact that shows what you learned. The artifact becomes both the analysis output and the input to the next design decision.

Counting (Quantitative)

Analysis through numbers — survey data, analytics, behavioral metrics. This mode provides scale and statistical confidence that qualitative methods cannot. It answers "how many" and "how often" after qualitative research has answered "what" and "why."

Counting works well when you need to validate qualitative findings with larger samples, prioritize among multiple opportunities by magnitude, or measure the impact of changes over time.

The three modes are complementary, not competing. The best analysis work moves fluidly between them: play to explore, make to crystallize, count to validate.

Mental Models

Erika Hall introduces mental models as a synthesis tool for understanding how users think about a domain ^just-enough. A mental model is a user's internal representation of how something works — their conceptual framework, not the system's actual architecture.

Understanding mental models matters because users interact with products through their mental models, not through your information architecture. If a user's mental model says "my documents are stored in folders," but your product uses tags, the user will struggle — even if tags are objectively superior.

To surface mental models:

• Card sorting: Give users cards with content items and ask them to organize them into groups. The groupings reveal their conceptual categories.
• Tree testing: Give users a navigation structure and ask them to find specific items. Success rates reveal whether your structure matches their mental model.
• Open-ended interviews: Ask users to describe how they think about a process or system in their own words. Listen for metaphors — they are windows into mental models.

Document mental models as diagrams showing the user's conceptual structure. Compare them to your product's structure. The gaps between the two are your usability risks.

Gap Analysis

Nishiguchi describes a form of gap analysis that compares behavioral data (what customers do) with psychological data (what customers think and feel) across different customer segments ^stck.

The method: for each segment, map both the observable behaviors (purchase patterns, usage frequency, feature adoption) and the psychological states (attitudes, motivations, perceptions gathered through interviews or surveys). Then look for gaps:

• Behavior-attitude gap: Customers who use the product frequently but express dissatisfaction (retention risk). Customers who express enthusiasm but rarely use the product (activation problem).
• Cross-segment gap: Segments that behave similarly but have different motivations (may need different messaging). Segments with similar attitudes but different behaviors (may face different barriers).

These gaps are where the most actionable insights hide. Aligned data (happy customers who use the product a lot) confirms what you already know. Misaligned data (unhappy customers who keep using the product anyway — probably because they have no alternative) reveals opportunities and risks that surface-level analysis misses.

Experience Map as Synthesis Tool

Torres describes the experience map as a tool for synthesizing customer interview data across multiple conversations ^cdh. An experience map plots the customer's journey through a problem space — not through your product, but through the real-world situation your product addresses.

The map captures:

• Steps in the customer's current process
• Pain points at each step
• Workarounds they have developed
• Emotional states (frustration, confusion, satisfaction)
• Opportunities for intervention

As you interview more customers, you layer their experiences onto the same map, noting where they converge and diverge. The result is a composite picture of the problem space that no single interview could provide.

The experience map feeds directly into the Opportunity Solution Tree, where each pain point or opportunity becomes a node that the team can explore with solution ideas and assumption tests.

Task Analysis and Workflow Decomposition

For products that support complex workflows, task analysis provides a structured method for understanding what users actually do ^just-enough.

Task analysis breaks a high-level goal into its component tasks, subtasks, and steps. For each step, document:

• What the user is trying to accomplish
• What information they need
• What tools or resources they use
• What decisions they make
• Where errors or delays occur

This decomposition reveals the moments that matter most: the decision points where users need better information, the handoffs where data is lost, and the repetitive steps where automation would save meaningful time.

Task analysis is particularly valuable when your research has identified a broad opportunity ("users struggle with reporting") and you need to pinpoint where to intervene. The decomposition moves you from a vague opportunity to a specific, addressable problem.

Quantitative Analysis

Quantitative methods provide the scale and statistical rigor that qualitative methods lack ^just-enough. But they require their own discipline.

Surveys

Surveys measure attitudes, preferences, and self-reported behaviors across a large sample. They are best used to quantify patterns already identified through qualitative research — not to discover patterns from scratch. A survey that asks "What is your biggest challenge with X?" will produce a list of surface-level answers. A survey that asks "How often do you encounter [specific problem identified in interviews]?" will produce useful quantification.

Design rules: keep surveys short (under 10 minutes), avoid leading questions, use established scales when possible (Likert, SUS), pilot with a small group before full deployment, and be rigorous about sampling. See Survey Design for detailed guidance.

Analytics and Event Tracking

Product analytics show what users actually do, not what they say they do ^{prr stck}. Instrument key events (sign-ups, feature usage, drop-off points, completion rates) and analyze them to identify behavioral patterns.

Cohort analysis groups users by a shared characteristic (signup date, acquisition channel, pricing tier) and tracks their behavior over time. It answers questions like: "Do users who discover feature X in their first week retain better than those who don't?" and "Has our onboarding redesign improved 30-day activation?"

Funnel analysis maps the sequence of steps leading to a desired outcome and identifies where users drop off. The drop-off points are hypotheses about where friction exists — hypotheses that qualitative research can then investigate.

Split Testing (A/B Testing)

A/B testing is the gold standard for causal claims: "Version B increased conversion by 12% relative to Version A." It requires sufficient traffic, a clear success metric, and statistical discipline about sample size and runtime ^just-enough.

A/B testing answers "which is better" but not "why." Pair it with qualitative research to understand the mechanism behind the result. A 12% conversion lift is actionable data. Understanding why Version B converts better is knowledge that informs every future decision.

The Dangers of Quantitative Analysis

Numbers feel objective. That feeling is dangerous.

Surrogation

Surrogation occurs when a metric intended to represent a goal becomes the goal itself ^just-enough. Net Promoter Score is intended to represent customer loyalty. When teams optimize for NPS rather than loyalty — gaming the score through survey timing, selective distribution, or post-interaction pleading — they have surrogated. The number goes up; the customer experience does not.

Vanity Metrics

Vanity metrics are numbers that look impressive but do not inform decisions ^just-enough. Total registered users, page views, app downloads — these feel good in board presentations but tell you nothing about whether your product is working. Prefer actionable metrics: active users, completion rates, time-to-value, retention by cohort.

The Local Maximum

Analytics can tell you which version of a button converts better. They cannot tell you whether you should have a button at all. Quantitative optimization finds the local maximum — the best version of the current approach. Qualitative research is how you discover that a fundamentally different approach would produce a higher maximum. Teams that rely exclusively on A/B testing risk perfecting something that should be replaced.

NPS: What It Measures and What It Doesn't

Net Promoter Score is the most widely used customer satisfaction metric and one of the most misunderstood ^{just-enough stck de}.

What NPS measures: A single-question proxy for customer sentiment and word-of-mouth potential. "How likely are you to recommend this product to a friend or colleague?" on a 0-10 scale.

What NPS does not measure: Why the score is what it is. What to do about it. Whether "promoters" actually promote. Whether the score predicts retention in your specific market. Whether the score is comparable across industries or cultures.

NPS is useful as a trend indicator — tracking the score over time within the same customer base reveals directional changes in sentiment. It is dangerous as an absolute measure or a comparison tool. A score of 40 in enterprise software means something very different from a score of 40 in consumer apps.

If you use NPS, always pair it with an open-ended follow-up question: "What is the primary reason for your score?" The qualitative responses are where the actionable insights live.

Reporting Findings

The worst thing that can happen to research is a long report that nobody reads.

Narrative Prototypes Over Reports

Lombardo and Bilgen advocate for narrative prototypes — findings delivered as stories, scenarios, or visual artifacts rather than bulleted lists of findings ^prr. Instead of a report that says "Users struggle with onboarding," build a storyboard that walks stakeholders through a specific user's onboarding experience, showing exactly where confusion arises and what the user does in response.

Narrative prototypes work because they engage empathy and memory in ways that bullet points do not. Stakeholders remember the story of Sarah, the marketing manager who spent forty-five minutes trying to import her contacts, long after they have forgotten slide 17 of the findings deck.

Showing Your Work via the OST

Torres recommends using the Opportunity Solution Tree (OST) as the primary vehicle for communicating research findings ^cdh. The OST makes the logical structure of your research visible: here is the desired outcome, here are the opportunities we discovered through research, here are the solutions we are exploring, and here are the assumption tests we are running.

See Opportunity Solution Trees for the full framework.

The OST serves as a living document that evolves with each research cycle. Stakeholders can see how new findings fit into the existing picture, which makes research feel cumulative rather than episodic. Each interview updates the tree rather than producing an isolated report.

Practical Reporting Principles

• Lead with the decision. "Based on our research, we recommend X" at the top of the page, with supporting evidence below.
• Use the customer's words. Direct quotes are more persuasive than researcher paraphrases.
• Show the evidence, not just the conclusion. Include enough raw data (quotes, screenshots, video clips) that a skeptical stakeholder can verify your interpretation.
• Keep it short. One page is better than ten. If you need more, create a one-page summary with a link to the full evidence.

The "Satisfying Click"

How do you know when analysis is done? Erika Hall describes it as a "satisfying click" — the feeling that pieces have fallen into place and the picture makes sense ^just-enough. Researchers call this saturation: the point where new data confirms existing patterns rather than revealing new ones.

Saturation is not a fixed number. For a narrow question ("Do users understand this label?"), five interviews may suffice. For a broad question ("How do small businesses manage their finances?"), fifteen or twenty may be needed. The signal is repetition: when you can predict what the next participant will say, you have probably heard enough.

The more useful guide is the quality of your decision confidence. If your team can clearly articulate what to build next and why, and everyone agrees on the reasoning, analysis has done its job. If there is still ambiguity about what the data means, you either need more data or a different synthesis approach.

Common Analysis Traps

Confirmation Bias in Analysis

Confirmation bias is dangerous during data collection, but it is equally dangerous during analysis ^{cdh prr}. During synthesis, it manifests as:

• Remembering the quotes that support your preferred solution while forgetting the ones that contradict it
• Clustering data into categories that confirm your hypothesis while ignoring clusters that challenge it
• Weighting the opinions of articulate participants over inarticulate ones, when the inarticulate participant may better represent the target user

Countermeasures: involve multiple team members in synthesis, explicitly search for disconfirming evidence, and require that any conclusion be supported by evidence from at least three independent sources.

Cherry-Picking Data

Cherry-picking is the cousin of confirmation bias. It involves selectively presenting data points that support a conclusion while omitting data that contradicts it. It can be deliberate (advocacy disguised as research) or accidental (the memorable quote overshadowing the boring-but-representative one).

Countermeasure: before presenting findings, ask yourself "What evidence did I leave out, and would including it change the conclusion?" If the answer is yes, your analysis is incomplete.

Analysis Paralysis

The opposite of premature conclusions. Analysis paralysis occurs when the team keeps gathering and analyzing data, unable to commit to a decision because they might be wrong. It is often driven by organizational incentives — if the cost of a wrong decision is punished more severely than the cost of no decision, teams will analyze indefinitely.

Countermeasures: set a decision deadline before you begin analysis. Define what "enough" looks like in advance: "We will interview eight customers and make a decision by Friday." Accept that decisions made with imperfect information can be revised as you learn more. The cost of delay is real, even when it is harder to measure than the cost of error.

Averaging Away Insight

When you aggregate qualitative data into summaries, you lose the outliers — and the outliers are often the most interesting signals. The customer who uses your product in a completely unexpected way may be showing you your next market. The one person who hated a feature that everyone else loved may be revealing a segment you have not yet identified.

Preserve the extremes. Note them alongside the patterns. The patterns tell you what is true on average. The extremes tell you what could be true.