9 comments

  • Dave HawkinsDave Hawkins, over 3 years ago

    I follow NNG's approach https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

    4 points
  • Luca MilanLuca Milan, over 3 years ago

    On my experience i can say that 6 people are enough. If you can have more people available you can divide people in bunch of 5-6 and make round of user tests trying the most critical parts learning from the previous tests.

    More people you test with the same set of tasks more time the analysis part will take.

    3 points
  • Sjors TimmerSjors Timmer, over 3 years ago

    A good answer I heard, is keep testing until you discover nothing surprising anymore

    2 points
  • Ryan Martin, over 3 years ago

    I use 5 as the magic number for qualitative product testing but some stuff I've been re-thinking.

    Is 5 still the magic number when you're doing generative research? (i.e finding problems to solve)

    Is 5 still the magic number when you have a diverse user group (i.e TV app users)

    Is 5 still the right number when you're investigating motivation to do something rather than the ability to do it? (this is probably where quantitative/MVP is preferred)

    2 points
  • Matthew SaforrianMatthew Saforrian, over 3 years ago

    This is a pretty solid recommendation. For those who are more interested in the details, there is some nuance here.

    In most cases, we as UX'ers are talking about Formative (what mistakes will users make) versus Summative (statistically, how many people will succeed).

    As you point out, the general consensus is a population size of 5 will deliver the best cost to benefit ratio. There are two scenarios where that doesn't necessarily hold true.

    The first, is if you only get one round of usability testing. While unfortunate, it happens–In that case, go for more as it's your only shot.

    The second, is when the probability of finding a mistake is very low. The following chart shows the chances of detecting a problem given the sample size and probability of seeing event once (think of dice, d3 versus d20).

    Jeff Gothelf, Quantifying the User Experince

    You can read the original magic number five article here if you want. Also, since everyone cites Nielsen I think it's prudent to point out that some of the credit should go to Bob Virzi and James Lewis who did a bunch of earlier work on this topic.

    I'd be happy to go into the nuance of the topic of Summative sample sizes but that gets deep into statistics which is no fun to write up...

    1 point
  • Ix TechauIx Techau, over 3 years ago

    I prefer X people, where X = as many as possible.

    1 point
  • Mak Raz, over 3 years ago

    the more the better

    0 points
  • Florent AlixFlorent Alix, over 3 years ago (edited over 3 years ago )

    Well, in my agency, we do a lot of user testing (I mean, there is rarely a project without user research and testing).

    My recommendations for one-to-one interviews would be : 6 is the extra minimum, 8 is nice and 12 is actually the maximum. above 12, you will rarely find anything interesting or unheard before. most of the time I try to do around 6 to 8 people of the target audience.

    of course, it depends of the maturity of the project and what you want to test. Is it the overall flow ? is it tiny graphical details ? or is it information architecture ?

    I like the idea of Luca to make rounds of 5-6 testers, it should return interesting things. (Also, if you want to do A/B testing, as many users as possible would be a must !)

    0 points
  • Mike Sarcone-Roach, over 3 years ago

    5 people is fine for 'Usability Testing', but 'User Testing' is a different beast. You want to run as many tests with as many users as possible, in as many realistic user scenarios as possible, and continue to do so on an ongoing basis.

    0 points