The One-Sided PMF Study
A product team wants to know whether they have product-market fit. They take one of two routes. Some recruit their power users, the ones who answer emails and turn up to the customer advisory board, and come back with a deck full of love. Others, usually after a bad quarter, recruit churned accounts and come back with a deck full of complaints. Both decks are honest. Both are wrong in the way that matters.
We call this the One-Sided PMF Study: a product-market fit investigation that samples only one side of the retention outcome, so it cannot tell the attributes that cause retention apart from the attributes that every user shares.
The mechanism is simple once you spell it out. If retained users say "the reporting dashboard is what I rely on", you don't know whether the dashboard keeps them or whether churned users relied on it just as much before they left for other reasons. If churned users say "it was too expensive", you don't know whether retained users feel the same price pain and stay anyway because something else outweighs it. A finding in a single sample is a description. A difference between samples is a cause worth chasing.
Why each sample lies on its own
Retained users rationalise. Someone who has used a product for eighteen months has built their workflow around it. Asked why they stay, they describe the product as it fits them now, not the moment it first earned a place. They name the features they touch most, which aren't always the features that made them commit. They also have a reason to be consistent: admitting the product barely clears the bar makes their own continued use look odd.
Churned users rationalise too, the other way. Leaving is a decision people justify after the fact. "Too expensive" and "missing feature X" are socially easy exits. They are often true. They are rarely the whole causal story. A churned user who never got past setup will still cite price, because price is the reason that doesn't make them look like they gave up. As our piece on the timestamp illusion in retrospective journey mapping shows, people rebuild sequences backward from where they ended up.
Each group's story is internally coherent. That is the danger. Coherent stories survive readouts.
What the contrast actually reveals
Running both samples with one shared guide turns five vague PMF questions into answerable comparisons.
1. Value: what is present in one group and absent in the other
The question isn't "what do people value?" It's "what value did retained users get that churned users never reached?" In one study for a mid-size B2B analytics product, both groups praised the dashboards in nearly identical words. The difference sat elsewhere: retained users had all, without prompting, described sharing a report with their manager in the first month and getting a reaction. Churned users had built reports nobody else saw. The dashboard was table stakes. Being seen to produce something by someone senior was the value. A retained-only study would have doubled down on dashboards.
2. Expectation gaps: same promise, different interpretation
Ask both groups what they expected when they signed up. Retained and churned users often heard the same marketing claim and understood it differently. "Automated insights" meant "it drafts the analysis for me" to one group and "it does the analysis for me" to the other. The second group churned at a much higher rate because the product delivered the first meaning. That isn't a product gap. It's a positioning gap, and the fix lives on the landing page, not in the sprint backlog.
3. Habit formation: the trigger, not the frequency
Retained users can usually name a recurring trigger -- a Monday standup, a weekly client report, a month-end close -- that the product attached itself to. Churned users describe using it "when I remembered". Frequency of use doesn't tell you this; two users logging in three times a week can have entirely different relationships with the product. Ask both groups: "What was happening right before you last opened it?" The contrast in answers is your habit map.
4. Alternatives: what each group compares you to
Retained users compare you to what they used before. Churned users compare you to what they use now. Those are different competitive sets, and the second one is the one that matters for retention. Frequently the "competitor" churned users name isn't a vendor at all -- it's a spreadsheet, an intern, or simply not doing the task. If the real alternative is doing nothing, no feature parity work will save you; the job wasn't urgent enough.
5. Activation: where the paths split
This is the most decision-useful contrast. Walk both groups through their first two weeks in detail and line the timelines up. There's almost always a point where the paths split: the retained user connected a data source on day two; the churned user waited for IT and never came back. That point is your real activation event, and it is frequently not the one in your product analytics dashboard. As in silent failures in agentic systems, a green success metric -- "completed onboarding" -- can hide the step that actually failed.
Designing a contrasting-sample PMF study
Sample on behaviour, not on self-report
Define retained and churned from product data, not screener answers. Retained means something like "active in at least three of the last four weeks, beyond 90 days tenure". Churned means cancelled or inactive for 30+ days. Add a third group if you can: at-risk users whose usage is falling. They're still close enough to the decision to remember it clearly and haven't yet rewritten it.
Match on everything except the outcome
The contrast only works if the groups are comparable. Match on segment, company size, acquisition channel and signup cohort. If your retained users are mostly enterprise and churned users are mostly self-serve, you're comparing segments, not outcomes. This is the same failure we describe in the persona collapse problem, applied to retention: blend unlike groups and the real differences disappear into the average.
Use one guide, word for word
Every question must go to both groups in the same form. The temptation is to ask retained users "what do you love?" and churned users "what went wrong?". Don't. Those questions produce mirror-image answers that can't be compared. Ask both: "Take me back to the week you signed up. What were you trying to get done?" and "Tell me about the last time you used it. What happened just before?" Neutral, event-anchored questions are the only ones that compare cleanly.
Interview churned users fast
The churn story degrades within weeks. After two months, "it didn't fit our workflow" is all that's left. Trigger the invitation within 7-14 days of cancellation. This is one place AI-moderated interviews earn their keep: an asynchronous interview that goes out automatically on cancellation reaches people while the memory is still specific, and at a volume a human team cannot schedule.
Aim for 12-15 per group, minimum
Contrast needs enough in each cell to see a pattern rather than a personality. With eight per side, one articulate outlier can flip a theme. With fifteen, you can say "eleven of fifteen retained users described a manager reaction in month one; two of fifteen churned users did" -- and that sentence is worth more than any quote. Our guide to using prevalence in qualitative research without faking quant covers how to report counts like this honestly.
Analysing for difference, not for themes
Standard thematic analysis across the whole pool will bury the signal. Pooled, the top theme will be something both groups say -- "reporting is useful", "price is a concern" -- because shared themes are, by definition, the most frequent.
Analyse differently:
- Code both groups with one codebook, built from the combined data, so the same idea gets the same label regardless of who said it.
- Build a contrast matrix. Rows are codes; columns are retained, at-risk and churned; cells show how many participants mentioned the code. Then sort by difference, not total.
- Treat shared themes as hygiene. If both groups say it, it's probably not why anyone stays or leaves. It might still matter -- a shared complaint about price is a ceiling -- but it doesn't explain the split.
- Look hard at asymmetric absences. The most revealing finding is often something one group says spontaneously and the other never mentions at all.
- Read churned requests carefully. "If it had X, I'd have stayed" is a classic case of the feature request literalism trap. Check whether retained users have X-shaped workarounds. If they don't need X either, the request is a justification, not a cause.
The decision this changes
A 40-person SaaS team we worked alongside had planned a quarter of integrations work, based on churned users repeatedly citing missing connectors. The contrasting study found retained users were missing the same connectors and staying anyway. The split was earlier: retained users had been onboarded by a colleague who already used the product; churned users had signed up alone. The integrations roadmap shrank to two connectors. The quarter went to a team-invite flow and a guided first report. That decision does not exist in a one-sided study, because neither side alone could have shown that connectors weren't the difference.
That's the point of the contrast. PMF isn't a feeling your best users have. It is the gap between the people who stayed and the people who didn't, and you can only measure a gap if you stand on both sides of it.
If you want to run contrasting-sample PMF interviews -- automated churn-triggered sessions, one guide across segments, and contrast-matrix analysis across groups -- book a Qualz.ai information session and we'll walk through a study design with your own retention definitions.
Practical takeaways
- Never run a PMF study on one side of retention. Recruit retained and churned users, and at-risk users if you can, under one design.
- Define groups from product data, with explicit thresholds for tenure and activity, not self-reported satisfaction.
- Match groups on segment, channel and cohort so the only systematic difference is the outcome.
- Use one neutral, event-anchored guide for everyone. No "what do you love" or "what went wrong" framings.
- Interview churned users within two weeks of leaving, ideally with an automated trigger on cancellation.
- Analyse with a contrast matrix sorted by difference. Treat shared themes as hygiene and asymmetric themes as candidate causes.
- Line up first-two-week timelines across groups to find the real activation event, and compare it against the one your analytics tracks.
