User Simulation Primer

A primer on
user simulation

User simulation means asking simulated people, instead of real ones, what they think of a feature, a price or a message. Answers come back in minutes or hours, not weeks, so you can test ideas that would never have justified a study. This primer is for product and marketing teams deciding whether to use it.

It goes by at least eight names, and the evidence on it looks contradictory. Providers publish accuracy figures as high as 96%, but a mega-study of 19 pre-registered studies found digital twins only slightly better than a model told nothing about the person. Both can be true, because they measure different things on different kinds of model.

This primer explains how user simulation works, how accurate it is, and what it’s good for.

In this primer
  1. 00 Introduction The names, how it works, and how accurate it is
  2. 01 Data What the models are built from
  3. 02 Modelling How the models are built
  4. 03 Accuracy How close the answers are
  5. 04 Uses What user simulation is for
  6. 05 History From life tables to language models

Semilattice, a user simulation provider, wrote this primer. We’ve tried to keep it objective. It covers the methods and the published research, not any one product, and the providers are compared on a separate page.

The names

Why there are so many names

Most of these names come from different fields but describe the same kind of product. Whatever a product is called, what decides its accuracy is how it is built.

Names what you get

User simulation
Product and marketing
Simulated people stand in for a product’s real users, or the audience for a message, and answer questions or use the product.
Synthetic respondents
Market research
Simulated people who answer a survey.
Synthetic users
UX and product research
Simulated users of a product, answering questions or using software. Synthetic Users is also the name of one product.
Synthetic data
Statistics and machine learning
Answers produced by a model instead of collected from people. The term also covers artificial copies of real datasets, made to protect people’s privacy.

Names how it’s built

AI personas
UX and marketing
A description of a person written into the prompt of a general-purpose model.
Silicon samples
Political science
A language model given demographic backstories to stand in for a survey sample.
Argyle et al., 2023
Generative agents
Computer science
Language-model agents that keep a memory and act as a character. Recent work has them act as specific real people.
Park et al., 2023; Park et al. (Simile), 2024

Names who is simulated

Digital twins
Marketing science and engineering
A model of one specific real person, built from their own answers.
Toubia et al., 2025

* The term this primer uses.

The Modelling chapter explains the three ways a model is built.

How it works

How a simulation is built and checked

Most products follow these four steps. Where they differ is the data in step 1 and the method in step 2. Those two choices decide how close the answers come in step 4.

  1. 1People

    Data about real people, such as survey answers, interviews, reviews or purchases. Some tools skip this step and rely only on what a general-purpose AI model already knows.

  2. 2Model

    A model is given a description of a person, or one real person’s data, or is trained on many people’s data.

  3. Example
    3Ask

    You put a new question to the simulated people. Each answers as one person, and the answers are counted like a survey.

  4. Example
    4Check

    To check accuracy, simulated people answer questions that real people have already answered. The model never sees the real answers. Then the two are compared.

Accuracy

No single number says how accurate it is

Providers publish accuracy figures as high as 96%, and research papers report 0.72 to 0.78. The figures measure different things, often on different kinds of model.

There are two main kinds of accuracy measure Some scores check each simulated person against a real one. Others only check that the percentages match, for example that 30% of the simulated audience says yes when 30% of the real one did. A model can get the percentages right while getting most individuals wrong, so the two kinds of score can’t be compared. Read more Most research papers use ordinal alignment It scores how close each person’s simulated answer is to their real one. Models reach 0.72 to 0.78. Random answers reach about 0.6, and real people asked the same questions again reach 0.82. Read more Surveys of real people have errors too People give different answers when asked the same question twice. Online opt-in panels miss official figures by about 6 points on average. Read more Each critique tested one way of building a model Most critiques have only tested one method. Check which method a critique tested before applying it to a product. Read more

The Accuracy chapter covers each point in full, with sources.

Chapters

What each chapter finds

  1. 01 Data What the models are built from Some data records what people say, some records who they are, and some records what they do. Each kind has gaps, and collecting more about each person does not always make a model more accurate.
  2. 02 Modelling How the models are built There are three ways to build a model: describe a person in a prompt, give the model one real person’s data, or train it on many people’s data. Some providers combine two of them.
  3. 03 Accuracy How close the answers are One system can be called 68.85% or 85% accurate, depending on how you measure it. On the score research papers use most, models reach 0.72 to 0.78 for each person. Real people asked the same questions twice reach 0.82.
  4. 04 Uses What user simulation is for It has three main uses: making the research you already do cheaper, testing decisions too small or too many to research with real people, and letting software try many versions of something and keep the best.
  5. 05 History From life tables to language models Statisticians have predicted populations since 1662. After 2020, language models made it possible to try predicting each individual.