Why you often need more than five users for usability testing

In addition to recruiting respondents, the total number of usability tests that you will perform is of great importance. In this article we look at what the literature says about it and we give you an indication of the desired number of respondents.

Much has been written on the Internet about the number of users you need to perform usability tests. Nearly all articles refer to the research conducted by the Nielsen Norman Group in 2000, which states that five users discover 80% of usability problems.

In this article we zoom in deeper on the study itself and find out why you often need more than five users to achieve good results.

From the graph and formula above, you get two interesting findings: 1) five users discover 80% of all usability problems 2) fifteen users discover all usability problems.

But ... What is often overlooked by UX professionals is the Nielsen's formula is only valid when similar tasks are performed by similar users. This means that the formula does not apply when:

• You want to test the user experience on different devices (mobile, desktop, tablet).
• You want to test the user experience in one large or different target groups. For example, a user test with users aged between 25 and 35 says little about the user experience for people over 50.
• The intentions of the users differ. Visiting a website for the purpose of gathering information or requesting a quote are two different types of user groups.

Although Nielsen's research was known to many people in 2000, the studies that question Nielsen's reliability are much less so.

For example, Spool & Schroeder (2001) establish that with five users only 35% of all problems are discovered. Perfetti & Landesman (2002) show that usability testing with more than five users is of great value and an average of five new problems are discovered for each additional user (up to 18 users).

Laura Falkner (2003) then takes this a step further by testing Nielsen's formula.

She did this by taking sixty usability tests, in which the testers have discovered all usability problems. When Nielsen's formula is correct, any group of five testers would find 80% discovering all those problems. Let's see what the results were. 

And guess what? Five usability tests detected not the promised 80% but 55% of all problems. Only when the number of users is doubled to ten, 80% of all problems surface and with twenty users you discover at least 95% of all problems.

Quite the difference! 

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.

As mentioned in 'When do I have to test the usability of my website, app or prototype?' you are better off conducting usability tests as early as possible in the development cycle. Usability testing should not be a once-off thing, but part of an continuous workflow. 

Based on the above studies, we recommend performing at least ten usability tests per device type (mobile, desktop and tablet). However, the exact number of usability tests you need depends on many different factors: 

• The budget that is available
• The amount of tasks that the user must perform
• Whether the results should be statistically significant
• Whether usability testing is applied once or more often in the design cycle
• The size of the target group (s)

In any case, you are always better off conducting even a small amount of usability tests. Remember: no testing, no insights.