Synthetic AI users repeat human conclusion, but with less variation

The question Nielsen Norman Group repeated with AI

In February 2026, the Nielsen Norman Group (NN/g) took an open-ended question from an earlier survey on UX terminology, answered by 604 technology and design professionals, and applied it to 120 synthetic respondents generated by the Synthetic Users platform. The question was simple: which professional field best describes the work of surfacing user needs and turning them into product recommendations? The result was published in an article written by Rachel Banawa on October 16, 2026.

NN/g had already tested synthetic users in interviews before; this time the scope was narrower, a single closed research question in open-ended format. To generate the 120 profiles, the team provided Synthetic Users with the full research plan (inclusion and exclusion criteria, and the questions), and the platform automatically assigned age, location, occupation, cultural background, and Big Five personality traits to each respondent.

Two details escaped the researchers' control. The platform simulates interviews, so the synthetic respondents answered as if speaking to an interviewer, producing longer responses than the original human survey. It also sometimes reworded the question on its own, swapping, for example, "people's needs and experiences" for "needs, behaviors, and experiences." Qualitative research knows well that changing one word changes the answer, and this takes away the researcher's control over a variable that should be fixed.

Same conclusion, but with a funnel of responses

In both data sets, UX remained the dominant term for describing this type of work; no alternative term dethroned it. At that level, humans and synthetics matched. But the similarity stops there: 84% of the synthetics used UX-related terminology, against about 70% of the humans, a statistically significant difference.

This difference becomes stranger when you look at the composition of the samples. The human base was recruited in part through NN/g's own channels, such as the Practitioner Panel, which made it more familiar with UX jargon; the synthetic base was generated by quota to cover a wider range of professions, with proportionally fewer UX professionals. In other words, the synthetic sample had, on paper, less pro-UX bias, and yet it produced more pro-UX responses.

The technical name for what happened: mode collapse

The explanation proposed by NN/g comes from machine learning vocabulary.

This pattern resembles mode collapse — a machine-learning term describing a generative model that keeps producing a narrow set of outputs instead of the full range it was trained on.

Rachel Banawa, researcher at Nielsen Norman Group

In practice, humans used more than 50 distinct labels to describe the work, from "consumer behavior research" to "human factors engineering," most cited only once or twice. The synthetics used just over a dozen terms and produced none of the unusual labels that appeared in the human base, such as "digital management" or "business analysis." Even responding in interview format, with longer texts and more chance of mentioning related terms, the synthetics still converged less.

Job title still matters, just less

Within the synthetic base, the respondent's stated job title still influenced vocabulary: 45% of synthetic customer insights professionals used the word "customer" in their answer, versus 0% of UX synthetics; 30% of synthetic market research professionals used "market," also 0% among UX ones. This effect mirrors what happened in the human base, and shows that the AI did not completely ignore the professional profile assigned to each respondent.

The difference appears when comparing the same job title across the two bases: 93% of synthetic product managers used UX terminology, against 44% of human product managers. The AI did not cancel out the effect of job title, but it pulled everyone, regardless of role, more strongly toward the dominant term than the actual behavior of people in that role.

What changes for whoever decides research day to day

This result is directly relevant to whoever decides how to validate a product hypothesis. If the research question only needs to confirm what the most common pattern is, which term people use, which option wins, which general direction the audience prefers, the synthetic base reached the same answer as the human base with a fraction of the time and recruitment cost. It's a legitimate use: checking a broad hypothesis before committing research budget to real people.

The problem appears when the question depends on the long tail, those unusual terms, needs, or behaviors that appear once or twice in the sample but signal an emerging segment, an unmapped pain point, or an underrepresented audience. It's exactly this kind of signal that mode collapse erases first, because the model tends toward the center of the distribution it learned, not the edges where the exceptions that matter to whoever designs inclusive products live.

NN/g suggests trying to compensate for this by asking the model for more diverse responses, but warns that generated variety is not the same as representative variety: the model would need to produce the right kinds of unusual responses, in the right proportions in which they occur in the real population, and without human data for comparison there's no way to know whether that happened.

Where synthesis still doesn't replace people

NN/g's conclusion is direct, and it applies to anyone considering replacing research steps with synthetic users: research with synthetic users is still not user research, because no real user is being studied.

Treat synthetic findings as a starting point rather than as evidence of what users think, say, or do.

Rachel Banawa, Nielsen Norman Group

For product and design teams under pressure to speed up discovery, the headline that it "reached the same conclusion as humans" is tempting, but deserves a careful read: the study shows that AI converges well at the center of the distribution and fails precisely at the edges, where users with disabilities, atypical behaviors, and needs that still have no established name live. Using synthetics to generate a quick hypothesis or check a broad pattern is reasonable; using synthetics to validate a decision that affects specific subgroups, without ever hearing from a real person in those subgroups, is repeating the classic biased-sample error, just with a veneer of automation.