Thanks for your comment, Dan!

That's why I love getting feedback from real people. While our data suggests the welcome mat is a good idea (increased conversion rate), you remind us that it actually sucks.

The good thing is, we're already redesigning our blog to get rid of this ugly distraction and provide a more meaningful experience for our readers.

Thanks for the motivation to keep working on this now :)

What a great comment! Thank you, Matt :)

I'm aware of the aesthetic usability effect and I completely agree that your perception of something changes how you actually experience it.

In my opinion perceptions are even more important than what really is (because reality is merely an illusion [formed by perceptions] … but that's another story).

That said, I have to reconsider the suggestions in this article. I'm still convinced that user testing is more about observing behaviour than it is about asking for opinions, but I perceive the line between the two of them to become more and more blurry after thinking about your comment.

I will think about that some more, and more importantly, make some experiments and present you the evidence. I'm doing a lot of user testing right now and so it's the perfect timing for this :)

Once again, thank you very much for your comment. And just so you know: I've used UsabilityHub many times before (to test logo ideas for example), and I love it for what it is—or for what I perceive it to be :D

Hi Louis-André,

thanks for your comment!

It's not exactly one of these user-agnostic situations, but as you'll see when you watch the clips, it didn't much matter that the site was tested by a man.

What I mean is that the missing filters and the slow page loading speed would have affected men and women alike. And as I said in the article, the size chart might be irrelevant to most women, but it could be a big aid to people with kind but horribly misguided intents, who need help (which is exactly what the size chart should provide).

If I had not screwed up the scenario (by asking for size M), our user would not have had any problems with choosing a bra. And that's the goal in terms of a website's usability. Everyone should be able to use your site (this is true for most sites)—even if he or she is not directly addressed by the copy and therefor not the exact target audience.

Are you saying there is no particular knowledge or life experience that comes with the intent of buying a bra ?

No, but I want to say that this particular knowledge or life experience is not a prerequisite for buying a bra online. And that's why we thought it might be a vivid example to underline this; by testing a site that's obviously geared towards women with a male tester and show that even this yields insightful findings.

I like to think of it this way: if you test your site with someone who's obviously not your target audience, and even this person is able to use your site with ease, you can assume that people with more knowledge about the subject matter will have an even better experience.

This doesn't mean that you shouldn't test with your target audience too to see if that's actually true. It only means that instead of not doing much user testing, you could just test your site with anyone and already get useful insights.

Anyway, thank you for your really interesting questions. I hope I answered them without too much babbling :)

Stefan

Hi Anna,

thanks for your comment!

You're right, and I would also be interested in a follow-up test—or even better yet, many follow-up tests with women (but also men) to see how different people experience the site. Because even if you test only with your exact target audience, people will experience your site differently and you shouldn't draw too many conclusions from watching a single user (I know that you know this :)

Anyway, the goal of our teardowns is not to prove that user targeting is insignificant. The goal is to teach people how to do their own user testing and how to analyze the results . And since user targeting is usually expensive and one of the main reasons why people don't do enough testing, I thought it would be important to remind them that even experts say, that for most sites, it doesn't much matter who you test:

“The best-kept secret of usability testing is the extent to which it doesn’t much matter who you test. For most sites, all you really need are people who have used the Web enough to know the basics.” – Steve Krug, Author of Don't Make Me Think and Rocket Surgery Made Easy

With that being said, user targeting is not insignificant (I will change that in the article; thanks for this). However, most of the times user targeting is not really important.

“The fact that you’re 18 to 35 years old with a college degree does not cause you to buy a product. It may be correlated with the decision, but it doesn’t cause it.“ – Clayton Christensen, Author of The Innovator's Dilemma and The Innovator's Solution

What causes people to do something is their reason to do it (or how Clayton Christensen puts it, the job people want to get done). And I don't agree that buying a present for a friend is a laughably uncommon edge case that causes people to buy a product.

And to be honest, I think that "user testing always works" is a bold statement. And it's important because it demystifies the topic, and even though I see a lot of value in testing with representative users (especially at early stages when you want to validate your product idea), user targeting should not be a showstopper or anything that prevents you from testing your website with real people.

But still, you're making a great point when you say that it would be interesting to see how a woman experiences the site differently. If you want, I let one of our female users test the website and send you the video once it's ready.

Let me know if you're interested and if you have an idea for a more common use case or if you think we should keep the same scenario to better compare the sessions. It depends on what you want.

Please let me know, I'd be happy to investigate this topic together :)

If testers can throw out any results they don't agree with, it defeats the purpose of testing.

The purpose of testing (how it's described in the article) is not to understand people's opinions and what they agree or disagree with. It's to observe them and to draw your own conclusions based on other tests, your experience, professional judgement, and common sense. I think the danger of false positives is low, because usability tests are usually not your only source of knowledge (nor should they be – even if you're testing with your exact target audience).

Thanks for the idea about writing an article on when you'll need a target audience and when it's save to ignore it. I'll definitely start thinking about this in the future :)

Hi Mike,

you're right, Userbrain doesn't offer demographic targeting. There are 2 reasons for that:

1) We don't have enough qualified testers yet

2) It's not necessary for most websites (Userbrain is only for websites) to test with representative users

E-commerce sites are an obvious example, but there are many web-based services out there, that should really try work for everyone (from a usability perspective) which is why testing them with random web users just works.

You're probably right about the problems of testing an Android app with iPhone users or vice versa. I wasn't aware of platform differences when I decided to add the word "app" throughout the article. Thanks for pointing this out :)

One last word about the danger of false positives: I think the worst that could happen, is that you're getting no (or inconclusive) results if you're testing e.g. an analytics tool with someone who has no idea about it. Identifying non-existent problems is not really a problem in my opinion.

Thank you for reading the article and taking your time to write a comment. I really appreciate this :)

That's a great question!

Since Ryan's way of working involves someone to connect your design to a database, I suggest you team up with a colleague who's really good at web development.

When I switched from Photoshop to HTML and CSS, I worked together with Mathias, one of our co-founders at Simplease. He did all the heavy lifting and all I had to do was providing him with HTML and CSS. We were able to deliver working prototypes in days and would then spend the remaining weeks on getting the usability and aesthetics right (our clients liked this as well).

I'm not sure about how your work environment or team looks like, but I suggest you talk to the person who would usually code your mockups. Ask him or her to watch one of Ryan's talks (that's what I did with Mathias) and then just start working.

I think there are 3 areas you may want to focus on:

1) Front end development Learning the building blocks was essential for me. I suggest using (and studying) frameworks like Bootstrap and Pure. Even if you want to write every single line of code yourself, it's more than just interesting to see how some of the smartest people on this planet organise and structure their files.

2) Typography You can't work like Ryan without understanding typography. I've read quite a few books on this topic and the one I find most useful is Lesetypografie by Hans Peter Willberg and Friedrich Forssman. Unfortunately it's only available in German, but it's too good not to be mentioned. For English readers I suggest Thinking with Type by Ellen Lupton to begin with.

3) User interface design In one of his talks Ryan mentions how valuable Edward Tufte's books on the visual display of information had been to him. You may want to read them too – multiple times. They are very special books and it's a pleasure just to hold them in your hands while gazing at the beautiful examples.

I hope that's somehow useful and wish you all the best with this :)

I totally get what you are saying. Especially the part that people just can't help reading text, is something I can confirm from my own experience.

Actually, that's the reason why an old article (even with typos) is better than lorem ipsum. When you show your client a layout with lorem ipsum they will only comment on things like typography, colors, and other visual design cues. Why would you want feedback on these kind of things from clients?

I think there's no need to explain a general page layout to a client (who is not a designer). Most of them don't care about our craft as much as we do, and they're right to do so. It's our job to get the layout right (which means to support the content) and we shouldn't have to ask them for their opinion on some random page designs featuring lorem ipsum. Of course you want their approval, but they don't need to understand why you choose different design patterns that adjust to all kinds of texts.

If you show them for example an old article instead of filling a page with lorem ipsum, you could probably get them to talk about their content. It just so happens naturally, maybe because they realize that the article you're using is old and telling you how they plan to deliver new ones in the future.

It's important to use real content to keep the discussion on this rather than just visual design decisions. If you don't focus on content, you will find yourself in meaningless debates with clients who want you to make their logos bigger and buttons look more activating.

I think, we should not cultivate this kind of discussions with our clients if we don't have enough time for them. Maybe they have a slightly different taste than we do (or even worse, there are multiple stakeholders with different tastes). I would not show these people lorem ipsum mockups just do discuss why someone likes or dislikes a color or a typeface I've chosen.

But of course you're right, sometimes you need a placeholder text and you need it quick. That's where lorem ipsum will come in handy, but not produce great work, but to get shit done :)

Thank you for your comment!

Hi Jeff,

thank you for taking the time to share your opinions and experiences. I really appreciate this!

I can understand your two options and want to propose a third approach: instead of choosing between sitting around doing nothing and doing cargo designs (designing without content) you could actually try to get your head around your client's business.

You can start by looking at their current website to understand what they're doing. After you have a basic idea, you can try to propose some website copy (if there's no one else writing it). This way you will automatically encourage your clients to read and criticize your writing in order to A) provide you with better content or B) hire an expert to write it.

Either way, you will be able to use real content instead of lorem ipsum. You just have to use your imagination and produce something yourself (even if it's wrong). I realize that this goes far beyond what most of us see as design, but I also think it's exactly the kind of design work that matters more than any other one.

BTW: as a web designer you probably know more about websites than ordinary people (even if they are good at writing). I think it's our responsibility to get involved into the heavy lifting of creating great content, for it's only way for to deliver a great web design.

And of course you're right, lorem ipsum will never die. It's like one of those super villains in comics ;)

Hmm … it's actually the same with us too. It happens from time to time that a client doesn't deliver content on time. The way bigger problem though is the in-house projects you are talking about.

Most of the time these in-house project tend to fade into the background because we're so busy keeping deadlines on client projects. That's why I always write my own copy for in-house projects to kind of get them off the ground.

Thanks for sharing this observation!