Why five users find most of the problems

A startup spends four months building a pharmacy delivery app. Before launch, the founder says: "We can't afford user testing. We'd need hundreds of people."

So they launch without it. In the first week, most people who reach the prescription upload screen give up. Nobody on the team had noticed that the Upload button sat below the keyboard once people started typing their address, so they couldn't see it.

Any five customers, watched for twenty minutes each, would have found that problem on day one. This module is about testing designs with real people, and this lesson explains why you need far fewer people than most beginners think.

Words in this lesson

What a usability test is

Jakob Nielsen describes a basic think-aloud usability study as needing only three things:

1. Recruit representative users.
2. Give them representative tasks to perform.
3. Shut up and let the users do the talking.

In a thinking aloud test, participants use the design while saying their thoughts out loud as they go. Nielsen wrote in his 1993 book Usability Engineering that thinking aloud "may be the single most valuable usability engineering method", and in 2012 he said he still stood by that.

He lists why: it's cheap (no special equipment), robust (you'll still get reasonably good findings even if you facilitate imperfectly, as long as you don't blatantly bias people), flexible (from paper sketches to finished products), convincing (sceptical teams change their minds when they watch real customers struggle) and easy to learn.

Why five users?

In 2000, Nielsen published one of the most famous articles in UX: "Why You Only Need to Test with 5 Users."

It's based on research he did with Tom Landauer. The number of problems found with n users is:

N × (1 − (1 − L)ⁿ)

where N is the total number of usability problems in the design, and L is the share of problems a single user reveals. Across the many projects they studied, L averaged 31%.

Here's what that means in practice:

Why does it level off? Nielsen explains: the second person does some of the same things as the first, the third repeats things you've already seen twice, and so on. After the fifth user, he says, you're mostly observing the same findings again without learning much new.

Then why not test 15 people at once?

Nielsen's curve shows you need about 15 users to find nearly all the problems. But he recommends something smarter:

If you can afford 15 people, run 3 studies with 5 users each.

His reasoning:

1. The first study of five finds about 85% of the problems. You fix them.
2. Your fixes might not work, and might create new problems. The second study checks, and finds most of the remaining problems too.
3. The second study can also dig into deeper issues, like information architecture and task flow, which were hidden behind obvious surface problems the first time.
4. The third study checks the second round of fixes.

Nielsen's conclusion: a design improves much more from three studies of five people than from one study of fifteen. That's iterative design.

Why not just one person?

Nielsen gives two reasons not to test with just one person each time:

• One person might do something by accident, or in an unusual way, and mislead you. Even three people give you a sense of how behaviour varies.
• Every study has fixed setup costs, like planning and recruiting, so it's better to spread them across several people.

When five isn't the answer

In a later article, "How Many Test Users in a Usability Study?" (2012), Nielsen repeats the answer, 5, "except when it's not". The exceptions:

He also answers a common objection. "But our website has millions of users!" That doesn't change the sample size: finding out whether a design element is easy or hard doesn't depend on how many people use it.

What "different groups" means in your work

Think about whether your users will genuinely behave differently:

• A school fees app used by parents and by bursary staff does very different tasks: two groups, test 3–4 of each.
• A savings app used by students and market traders doing the same tasks: probably one group, but recruit a mix of both.

Five people, not just any five

The five-user rule only works if the five are representative: people who match your users (Module 2).

Testing with your classmates, your family or other designers tells you how they use it. For a pharmacy app used mostly by people collecting prescriptions for older relatives, those are the people to test with.

And remember, from Module 2, the Nigeria Data Protection Act, 2023: get informed consent, collect only what you need, and protect recordings and notes.

Test early, test cheap

Nielsen says thinking aloud works at any stage: paper sketches, Figma prototypes, or finished products.

The earlier you test, the cheaper the fixes. Moving a button in Figma takes a minute. Moving it after launch takes a developer, a release, and a week of lost customers.

Common mistakes

The short version

• A basic think-aloud test needs representative users, representative tasks, and a facilitator who lets them talk. Nielsen calls thinking aloud possibly the single most valuable usability method.
• Nielsen and Landauer: problems found = N × (1 − (1 − L)ⁿ), with L ≈ 31%. Five users find about 85% of problems.
• Better to run 3 studies of 5 users, fixing between each, than 1 study of 15.
• Exceptions: 3–4 per group for very different groups; 20+ for quantitative studies; 15+ for card sorting.
• The five must be representative. Get informed consent. Test early, when fixes are cheap.

Try it yourself

1. Using the formula, work out the share of problems found with 4 users when L = 31%. Show your working. (Hint: 1 − 0.69⁴.)
2. Look at your product. Does it have distinct user groups who do very different tasks? Decide how many people you need for one round of testing, and justify it with Nielsen's guidance.
3. Write a short recruitment description: who counts as representative for your product, and who doesn't.
4. Plan three rounds of testing on a simple timeline: what you'll test in each round, and when you'll fix things in between.
5. Find one person who matches your users, and do a quick, informal test of your Module 8 prototype: ask them to try one task while thinking aloud, and don't help. Write down everything that surprised you.

You'll know you've got it when you can explain to a sceptical client, with Nielsen's numbers, why five people and three rounds will teach them more than one big study, and you've already watched one real person use your design.