6 comments

  • Ben Patterson, almost 4 years ago

    Hoo have I heard these before, and not just from other people. Sometimes these are valid reasons, but you should always check yourself if you hear them.

    2 points
  • , almost 4 years ago

    I wrote this to give some tactical advice on being more assertive during product/design meetings. I'm also really excited because I created an illustration for the post - normally I'm god awful at drawing. But I spent a good amount of time on it and I'm happy with where it ended up :D

    2 points
  • Cihad TurhanCihad Turhan, almost 4 years ago

    You have collected very accurate phrases that we did or may hear during a product meeting. Maybe there are thousands more.

    However, I strongly suggest not doing them when you build your product at the first time. For example, no one should expect an A/B testing for an MVP product. Some features should be added in the mature period of the product not at the day one.

    0 points
  • Andrew Richardson, almost 4 years ago

    A/B tests consume a lot of time and effort. Do you really feel it’s worth investing that time? It may be a better use of our time to make a gut decision on this, or find proxy data that we can use now.

    Do they really? There are certainly instance where this is true but in my experience split testing is relatively low effort (2-3 hours of pitching/setup/reporting time for one person) and a good way to validate ideas and be sure that our "gut" is accurate. Especially in areas where user testing doesn't quite give that level of precision (i.e. what users say they do vs what they actually do).

    Otherwise this is a pretty good list but that one particular felt a bit short sighted.

    0 points
    • , almost 4 years ago

      Good point - I guess it really depends. Some A/B tests are super simple to run. A lot of the suggestions that I hear during product meetings are much higher effort though. For example, an entirely different layout of a page, or a different interaction metaphor for some UI. It also highly depends on the capabilities of your A/B testing framework. At Microsoft, literally every test was a huge amount of work (my team was the first team in Office to even do A/B testing...lolllllll). Conversely when working at Imgur, we used Optimizely and could test small changes super easily.

      0 points
      • Andrew C, almost 4 years ago

        A lot of A/B testing is statistically insignificant junk science. But that's just because the method, like any, can be abused. For me, user testing and research has usually proven a better bang for it's buck when it comes to making the right call.

        0 points