4 comments

  • Anna NiessAnna Niess, over 4 years ago (edited over 4 years ago )

    "We decided to test it with a male user to show you how insignificant user targeting is most of the time."

    If you are trying to prove that user targeting is insignificant, shouldn't you also test it with a female? If they browsed and experienced the site in the same way, you would be proving your point. But you just tested it with a dude, and the use case you assigned to him is a laughably uncommon edge case:

    Imagine you're looking for a birthday present for a friend. She's a runner and since she said that she's still looking for a sports bra, you figured that this might be a nice present for her.

    The article is trying to argue that "user testing is always works," which is sort of a non-statement. How valuable are the insights when testing an obscure made-up edge case? I'd be interested in a follow-up test with a woman. See how their sessions differ and how they are similar, and recommend design / ux changes based on common use cases.

    2 points
    • Stefan RösslerStefan Rössler, 4 years ago (edited 4 years ago )

      Hi Anna,

      thanks for your comment!

      You're right, and I would also be interested in a follow-up test—or even better yet, many follow-up tests with women (but also men) to see how different people experience the site. Because even if you test only with your exact target audience, people will experience your site differently and you shouldn't draw too many conclusions from watching a single user (I know that you know this :)

      Anyway, the goal of our teardowns is not to prove that user targeting is insignificant. The goal is to teach people how to do their own user testing and how to analyze the results . And since user targeting is usually expensive and one of the main reasons why people don't do enough testing, I thought it would be important to remind them that even experts say, that for most sites, it doesn't much matter who you test:

      “The best-kept secret of usability testing is the extent to which it doesn’t much matter who you test. For most sites, all you really need are people who have used the Web enough to know the basics.” – Steve Krug, Author of Don't Make Me Think and Rocket Surgery Made Easy

      With that being said, user targeting is not insignificant (I will change that in the article; thanks for this). However, most of the times user targeting is not really important.

      “The fact that you’re 18 to 35 years old with a college degree does not cause you to buy a product. It may be correlated with the decision, but it doesn’t cause it.“ – Clayton Christensen, Author of The Innovator's Dilemma and The Innovator's Solution

      What causes people to do something is their reason to do it (or how Clayton Christensen puts it, the job people want to get done). And I don't agree that buying a present for a friend is a laughably uncommon edge case that causes people to buy a product.

      And to be honest, I think that "user testing always works" is a bold statement. And it's important because it demystifies the topic, and even though I see a lot of value in testing with representative users (especially at early stages when you want to validate your product idea), user targeting should not be a showstopper or anything that prevents you from testing your website with real people.

      But still, you're making a great point when you say that it would be interesting to see how a woman experiences the site differently. If you want, I let one of our female users test the website and send you the video once it's ready.

      Let me know if you're interested and if you have an idea for a more common use case or if you think we should keep the same scenario to better compare the sessions. It depends on what you want.

      Please let me know, I'd be happy to investigate this topic together :)

      0 points
      • Louis-André LabadieLouis-André Labadie, 4 years ago

        The underlying assumption in your answer is that a site where you buy bras is one of these user-agnostic situations:

        [...] for most sites, it doesn't much matter who you test

        Are you saying there is no particular knowledge or life experience that comes with the intent of buying a bra ?

        Not, for example, the kind of knowledge that a man (with kind but horribly misguided intent) may not have, whereas almost any woman would?

        Or the kind of life experience to which the site's copy is addressed, from its very first sentence – but which may not make much sense to someone without it?

        1 point
        • Stefan RösslerStefan Rössler, 4 years ago

          Hi Louis-André,

          thanks for your comment!

          It's not exactly one of these user-agnostic situations, but as you'll see when you watch the clips, it didn't much matter that the site was tested by a man.

          What I mean is that the missing filters and the slow page loading speed would have affected men and women alike. And as I said in the article, the size chart might be irrelevant to most women, but it could be a big aid to people with kind but horribly misguided intents, who need help (which is exactly what the size chart should provide).

          If I had not screwed up the scenario (by asking for size M), our user would not have had any problems with choosing a bra. And that's the goal in terms of a website's usability. Everyone should be able to use your site (this is true for most sites)—even if he or she is not directly addressed by the copy and therefor not the exact target audience.

          Are you saying there is no particular knowledge or life experience that comes with the intent of buying a bra ?

          No, but I want to say that this particular knowledge or life experience is not a prerequisite for buying a bra online. And that's why we thought it might be a vivid example to underline this; by testing a site that's obviously geared towards women with a male tester and show that even this yields insightful findings.

          I like to think of it this way: if you test your site with someone who's obviously not your target audience, and even this person is able to use your site with ease, you can assume that people with more knowledge about the subject matter will have an even better experience.

          This doesn't mean that you shouldn't test with your target audience too to see if that's actually true. It only means that instead of not doing much user testing, you could just test your site with anyone and already get useful insights.

          Anyway, thank you for your really interesting questions. I hope I answered them without too much babbling :)

          Stefan

          0 points