A/B testing tool for designers

8 days ago from , UX Developer & Tech Lead

We know we can easily do A/B testing once designs are coded and they are live as a website or an app.

I'm wondering if there is a tool that allows designers to do A/B testing before a design is live. Something that potentially allows also to test non web designs (i.e: a/b test letter heads or pamphlets etc.)

22 comments

  • Ole Harland, 4 days ago

    Create two pictures and put them into a google form with two radio buttons. Send this link to your test candidates and done.

    4 points
    • Thomas PalumboThomas Palumbo, 3 days ago

      But what are you actually testing? Opinions on design or usability?

      2 points
      • , 3 days ago

        That depends on what I am testing. Sometime it is a matter of testing opinion if you are comparing say 20 different design of the same logo.

        Other times you might want comments on usibility of 2 different version of a same app.

        0 points
    • barry saundersbarry saunders, 3 days ago

      That'll test which ones people like, not which ones are effective.

      2 points
    • , 3 days ago

      Hey thanks Ole, I tried doing this with Typeform. But doing it with form presents some problems like image zooming.

      Comments on images etc.

      0 points
  • Andric ThamAndric Tham, 3 days ago

    Yes, there is. I’ve been using a tool called Chalkmark that lets you upload a bunch of static images to test “first clicks”. It’s been such a dream using it.

    It’s kind of like a usability test for mockups. You can give users a series of screens accompanied by a task, and then they have to click on wherever they feel helps them achieve that task.

    For example, if you want to test if users know where to click to checkout on an online shop, can have a task like: “You just added a pair of headphones to your cart and want to checkout. Where would you go to do so?”. You can set your cart icon to a “correct area”, and Chalkmark will record if users clicked on it or not, and give you a % score.

    To achieve an A/B test on a static design, simply upload the A version and B version(s) to Chalkmark as separate tests. Chalkmark will give you an option to recruit users from their panel (very cheap, compared to usertesting.com), or you can simply use the provided link and share it with your own contacts.

    To automate the process further, you can use Ethnio to intercept users on your site and send them randomly to different tests. You should also be able to build some kind of workflow or pipeline around the data that Chalkmark gives you since it’s a simple .xlsx file.

    3 points
    • , 3 days ago

      Hey Andric, thanks for the talking about Chalkmark. I'm keen to try Chalkmark out. It seems kind of pricy for one designer to get started though.

      But it definitely looks like something I'm looking for.

      0 points
  • Sam Bible, 21 hours ago

    Depending on what feedback you're looking for, A/B testing might not be the best tool.

    A vital concept of A/B testing that often gets overlooked is that you're going for statistical significance. Without statistical significance results are often little better than guessing. The catch is that it often requires a larger sample size depending on the variables involved, which is why they are commonly conducted with a live audience. There's a great overview here.

    But sometimes it's incredibly helpful to just get a quick read on preferences, potential trouble spots, what stands out to visitors, etc. A couple of tools I've found really useful for that in the past are:

    3 points
    • , 14 hours ago

      Hey Sam thanks for sharing the links of Zurb & Usibility Hub. They both look like great resources. I think what I'm after is more like the Preference Test option provided by Usibility Hub.

      0 points
  • brennan smith, 3 days ago

    I use UsabilityHub and get good feedback from each user tested.

    1 point
  • Paul ArmstrongPaul Armstrong, 3 days ago

    I run a team of 5, and for the amount of traffic we get and the type of product we have, we've gone with VWO.com.

    As was stated previously, the most important part of any A/B testing of variations is the data. Make sure you are eliminating as many variables against the most robust data available. It's just the scientific method:

    1. Hypothesis : What do you want to learn, and what do you believe will be affected by this change? (e.g. changing the color of this button will increase clicks).
    2. Data : Gather about 6 months of data from (a source like) Google Analytics around your hypothesis (what's the CTA of this button?).
    3. Test : create the variation (b) that will run against control (a).
    4. Conclusion : Review the results of the CTA on A and B
    5. Next steps : What things should you implement based on what you learned?
    1 point
  • chris caseychris casey, 14 hours ago

    usabilityhub.com

    Upload two designs and ask community which they prefer

    1 point
  • Billy CarlsonBilly Carlson, 2 hours ago

    I have long wanted to build a much simpler tool for this. I feel like the examples already posted are either too feature crazy, or with the usability hub, too simple. Another feature I would like for this, is allowing the user to view multiple versions of the same design, so they can see how it evolved over design iterations.

    1 point
    • , 1 hour ago

      Hey Billy, I totally agree with you. In fact this is one of the reasons I posted this question in the first place.

      I would like to build something for my team's internal use. Something that is not too feature heavy and possibly free for designers.

      Would you be interested to collaborate with me? Even if you do some early beta testing that might be helpful.

      0 points
      • Billy CarlsonBilly Carlson, 1 minute ago

        I would absolutely love to! I have been talking with my design team about creating wires and designs for this tool. Hit me up at billy [at] threadless dot com! It would be so fun if we could work something out.

        0 points
  • Joanna Jiang, 2 days ago

    Hi Bappy. The tool that you want sounds exactly like BetterDesign (www.betterdesign.io). I had the same question and I went online for a tool to solve it. However I couldn't find anything. So I decided to create it.

    Feel free to check it out and let me know your thoughts, cheers.

    Joanna

    1 point
    • , 2 days ago

      Hey Joanna, that is a great tool. I would love to learn more and also find out what the responses have been from the market for this tool for you.

      I'm wondering what will be the best way to contact you directly. I'll be awesome if we can jump on the phone for a few minutes.

      0 points
      • Joanna Jiang, 23 hours ago

        Hi Bappy Glad to hear that. You can contact me through email contact@betterdesign.io. let's exchange number there and schedule a call.

        We are running a private beta now. If you would like to test it out, I would be happy to send you an invite as well.

        0 points
  • V. MV. M, 3 days ago

    If you want to try A/B testing on the flow - you can try http://CanvasFlip.com . Records user videos, heatmap & even shows how conversions will look like..

    Hope it helps.

    [Disclaimer: I am associated with CanvasFlip.]

    0 points
  • Johannes IppenJohannes Ippen, 12 hours ago

    Hey Bappy, what you're looking for is not really an A/B Test but a survey – check out Google Consumer Surveys, they are great for that: https://www.google.com/analytics/surveys/

    0 points