• Matt RothenbergMatt Rothenberg, 5 years ago

    Your central thesis ("It’s busy-work, it’s a way to avoid making hard decisions") is predicated on what I would argue is a flawed assumption: that product teams who embark upon up-front user research have enough data to make these kinds of decisions.

    The one example that you provide in this blog post gives short shrift to generative user research, and creates a false dichotomy between it and the 4-5 day "Design Sprint." For instance, you claim:

    For the most part, though, I recommend ditching the time consuming, wasteful up-front research in favour of tangible results.

    In painting this dichotomy, I think you've left out a couple of important points:

    • There is a time and a place for up-front, generative user research. Your decision to use this technique should be a function of the fuzziness of the problem space, and not of how much cash you're looking to get from your client.
    • Not all up-front research initiatives look the way that you've described. At Pivotal Labs, for example, we gain plenty of "results" through: clarification of the problem space, a deeper understanding of users' needs, building/testing prototypes (of varying degrees of fidelity), and often a backlog full of user stories that engineers can start.
    • If your conclusion is that "the useful data came from the first user tests, not the research," I encourage you to ask yourself whether that's a failing on your part to conduct the research in an effective manner (choosing the right tool for the job) or whether that's a failing of generative research as a whole.

    From perusing the comments on Medium, it looks like we are in violent agreement on the matter. I worry, though, that headlines like "User Research is Overrated" create generalizations that impede our ability to have productive dialogue about our practice.

    10 points
    • Jonathan Courtney, 5 years ago

      Hey Matt, cheers for the thoughtful well-written response.

      Now let me get to it: The title of my article is very bullish, as is the way I speak about design in general. It's the exact opposite of your very pleasing-to-read scientific approach I would say. I'm writing/speaking from my subjective perspective based on what has worked for me and my company over the past 5 years.

      This means that yes, some of subtleties are lost in favour of hitting my point home that User Research, at least how it is being conducted in many of the most prestigious companies in the world, is very wasteful and could do with being a more tangible process that in-house teams can use rather than just agencies.

      Just because Pivotal Labs does it "right" doesn't mean I shouldn't expose the huge percentage of other companies who spend millions and months of waste on documentation they never use.

      Speaking about impeding our ability to have productive dialogues about our industry: This is only if people take themselves too seriously (like people in the design industry tend to). I'm happy to write what I think, then use that to open a dialogue like is happening right now.

      I'm sure that wasn't the most satisfying answer, but hope there was something in there :)

      Cheers, Jonathan

      1 point
      • Matt RothenbergMatt Rothenberg, 5 years ago

        Thanks for the response and clarification!

        By "impede our ability to have productive dialogue," I didn't mean to suggest that such a dialogue couldn't be achieved. I meant that it's far more difficult to see where exactly you're coming from and what your argument is.

        1 point
        • Jonathan Courtney, 5 years ago

          Ah, ok, yeah that's probably true. I'm gonna be honest here: usually nobody reads my articles, so I wasn't expecting to even have to explain all of this :D If you're ever in Berlin though, would love to meet and have a rant!

          Cheers, Jonathan

          1 point
      • Thomas Michael SemmlerThomas Michael Semmler, 5 years ago

        I am so sorry, I know it must seem like I troll you, I don't, but I can't let this sit there:

        It's the exact opposite of your very pleasing-to-read scientific approach I would say

        All content on design ever is blunt, assumptive and fluffy. Once in a while you get that very pleasing-to-read scientific approach - article, but that is extremely rare.

        • design with heart
        • why developers also need design inspiration
        • look at these redesign concepts that do literally nothing else but apply current trends
        • alternatives to hamburger menus
        • {concept} killed my {subjective way of doing things}
        • I {bad subjective experience}, here is why you should do the same

        Without sounding too salty, I wonder what bubble made you think that this is the exception, and very pleasing-to-read scientific approach is the majority - because I wanna get into that bubble! Please.

        1 point
      • Nat BuckleyNat Buckley, 5 years ago

        Design is about communication. If your goal is to communicate an idea, then the choice of how to do it matters. If the important subtleties of what you're trying to communicate are lost due to your bullish style, then it's worth considering alternative approaches.

        1 point
        • Jonathan Courtney, 5 years ago

          It's only lost on the people looking to be outraged on the internet, and I really couldn't care less about those type of people. Take what you want from articles you read online and discard what you don't like. It's very simple really.

          0 points
  • Sacha GreifSacha Greif, 5 years ago

    It was part of our UX/Product Design package: 2–6 weeks of up-front user research.

    I think one of the big divides people don't talk about enough in the design world is the difference between design agencies, who are always looking for new methodologies and angles to sell to their customers, and in-house design teams or individual freelancers, who usually just need to Get Stuff Done.

    It's kind of funny reading about fast-moving, don't-overthink-it "design sprints" promoted as some kind of breakthrough, when that's just how most small companies and startup operate out of necessity and lack of resources anyway…

    5 points
    • Jonathan Courtney, 5 years ago

      Hey Sacha, cheers for the comment.

      Let me just be bluntly honest here: If the in-house teams of the companies I work for were already in the "Get Stuff Done" mode - then I wouldn't be writing something like this. There's an insane amount of waste in the processes of most startups, large corporations and agencies, even if they claim to be running a lean process.

      So while I agree that there is a big divide between agencies and in-house teams, I still think a huge majority of in-house teams still need to assess just how much of their process is wasting time/money. Yes, even the ones strapped for resources.

      5 points
  • Marcel van Werkhoven, 5 years ago

    I say it's underrated. There's still plenty of design/usability problems on the web that could've been solved if 'anyone' outside the design team was asked what they thought about it before it was build.

    No company would build a car without thinking about the driver.

    Also user research doesn't have to be expensive. With a short interview a lot of trouble might be prevented afterwards. I do agree that's it pretty dumb to roll out the same type of research for every project. You might not always need to create personas and such.

    User research should be a part of the entire design process not just something to get out of the way before designing and building the product.

    And finally, even terrible research is valuable research. Since you've discovered a massive problem in your process that you're likely never going to repeat again.

    3 points
    • Jonathan Courtney, 5 years ago

      Hey Marcel,

      My basic point is that the research should be integrated into the production of the product rather than a specific, standalone step before. I mention that in the first sentence, but of course my headline is pretty blunt and might throw people off a little.

      Thanks for your comment.


      1 point
  • Thomas Michael SemmlerThomas Michael Semmler, 5 years ago

    Other answers already pointed out things that I would point out too, but what I say is, that you are still doing user research, after the testing that is. Its basically a gamble, because it can be very valuable, or potentially destroy your project entirely.

    1 point