Measuring the success of visual design

over 7 years ago from , Product design @Paperlesspost

I'm having a hard time figuring this out :

You're redesigning the navigation for your app. This is only a visual redesign. Refreshing the colors, type, layout. The information architecture does not change. There are 5 buttons / links on this nav, and they stay the same. I only change their layout around a bit and refresh the visuals.

Is there a way to measure this redesign?

Do I just look at the standard conversion metrics for the old nav vs new nav? Won't the conversions stay the same in the short term, since the information architecture is still the same?

Is it possible to measure the longer term, emotional impact of the redesign? (If the assumption is that this new nav "looks" cleaner, and is more "fun" to use, how can I ensure that this is actually the case with real world users and not just designer bias?

If your answer is to user test the nav, what kind of a user test would you run exactly? 5 people use the new nav vs 5 other people on the old nav? Or same 5 people use both the navs and give an opinion?

I would really love to know what schools of thought are there on this. Is it even worth measuring something like this?


  • Andrew Richardson, over 7 years ago

    Depends on what your goals are. Why did you change the navigation in the first place? Making it "cleaner" and more "fun" aren't easily measured. User testing is probably the best way to measure such things.

    It's possible to set up split tests to see how people navigate around but it's hard to draw firm conclusions from that.

    It seems to me like you might have been solving a problem that may or may not have existed. Which is something we all do. We start to think "Oh this isn't very attractive" so we change it. Then we want to test it and find out there wasn't much difference and we probably just wasted time.

    The more and more work I do the more I feel like our real job is to find the right problems. There is no shortage of people who can make things pretty but if it doesn't matter to the customer then what value are we providing?

    TLDR: Make sure you are testing to find problems first and then use that to dictate what should be changed and test against the old benchmark.

    2 points
  • Bady QbBady Qb, over 7 years ago

    In wego we do visual enhancement and test how it performs. We use optimizely to A/B test and see & measure how each changes performs. Sometimes we only changes label but most of the times we also changes visual component like CTA buttons with different color.

    most of the time it has impact to the conversion.

    1 point
  • Peter Stacho, over 7 years ago

    I'd check for variations in pageviews per visit, average time on site, bounce rate, and conversion rates. Sounds like a simple split test would be perfect for this if you enough traffic.

    Visual design can make a huge difference, and I can tell you from experience that sometimes designs we'd label as "shitty" actually converted better than the pixel perfect masterpieces we originally had in mind.

    1 point