What this described is Lean UX.
However, it's still very fundamental that people do not test their own designs. That bifurcation of responsibilities allows someone who is not tied to the concept test it without bias. You still have to negate bias, otherwise it will end up either as user-driven kludge, or pretty but unusable schlock.
One or two ad-hoc interactions with users with measurable outcomes does nothing to negate the fact that over the whole, a project will incur some measure of bias debt by having the designer test their own designs.
That's a valuable insight.
But isn't bias sometimes a good thing when it comes to design? I think about Apple and the UI designs they deliver for their mobile devices. They are very opinionated about this design and that's why so many people love it. Of course there are many who hate it as well, but that's kind of fine, since there are other solutions out there, built from other people with other opinions.
Talking about biases and opinions – aren't usability experts also biased in one or the other way. I mean they are still people and they still have their own beliefs, past experiences and opinions about everything. They are not as objective as one may think they are.
Bias can be a good thing - it's why you'd want a subject matter expert designing a system for Air Traffic controllers, instead of a logo designer, for example.
However, using someone else allows you to treat tests in a double-blind fashion. UX researchers are trained in methodologies that allow them to conduct their tests without providing leading information or questions. This process has dual intents: It lets the tested participant be open about their feedback, and it lets the researcher look at the design without emotion in order to evaluate. It's not uncommon for UX Researchers to come back with suggestions about how to change a UI, but it's only after they report back with their insights that their biases may be revealed. At that point, anything they present can be balanced against the raw data itself.
I think it's always interesting to see Apple brought into the conversation. There seems to be a belief that Apple's organizational bias towards a certain level of quality in design means that they're not respecting the long-held best practices of the User Centered Design process. From everything that I've seen in the way of their documentation, HIG guidelines, and from every Apple'r I've talked to, this is simply not the case. Apple has a heavy tendency to command-and-control in the production design process. Like a German football team, you can fully expect Apple's production design to be highly effective in execution and stay within the lines. However, that rigidity in execution does not correlate to a lack of proper UCD methodology in the ideation phase. It's because Apple is so far forward thinking that they're able to come up with different interaction and visual design patterns, and are able to refine them into something that seems effortless and intuitive. To be sure, they are testing.
Ultimately, all of this is supplemental to my original point. Lean UX advocates that a hypothesis is formulated, validated, and refactored as needed, as quickly as possible by taking bits of work in more manageable chunks. The difference is that Lean UX doesn't advocate cutting out the collaboration of the team. Instead of testing something yourself, your developer may test it. You may have a Researcher on hand to test it. No man is an island - and no one person will catch every issue with their design, even if they test it themselves.
Thank you for your insights. Very interesting observations :)
According to your experience with Lean UX, what's the more typical approach: everyone in the team doing some testing or someone doing all the testing for everyone else?
Or maybe a better phrasing of the same question: do you know designers who at least sometimes perform their own tests or is there a strict separation between design and usability as there used to be one (and sometimes still is) between design and development?
In my experience, it's been everyone involved in coming up with the solution doing the testing. Product managers, developers, and even users - if you're lucky enough to have them be a part of the collaborative team.
I feel like I may have oversimplified part of the Lean UX process, however. It's been my experience that in Lean UX, there is no "single wringable neck" for the design - the design is a derivative of the collaboration of the team.
Also important to note, the testing is done at an early enough stage to avoid any confirmation bias cropping up, hence the concept of "testing hypothesis" instead of "testing design," where design implies something a bit firmer.
In my current role, we've got a dedicated UX researcher who does our testing with remote/external participants, and then participates in larger design sessions to help shape the next iteration of a design. This model works well for us because she's fast in testing, analyzing, and reporting back what the pain points were. Additively, we're proactive about being part of the test planning and focusing on smaller chunks of functionality for testing.
I hope that helps to paint a better picture.
I also wanted to say, I can totally understand the place you were in when writing the original article - it's extremely tough dealing with outside agencies sometimes when you're an internal designer, but it's important to note that what those agencies provide back is objectivity, and it just another input into your process. It should never override your internal subject matter expertise or understanding of the user landscape.
Thanks again for your insights. It's a great way of learning for me :)
I'd like to ask you one final question: when you talk about "testing hypotheses" what exactly do you mean by that? Testing sketches, mockups, MVPs or maybe only marketing headlines via Google AdWords? Or something completely different? And is it a qualitative or more quantitative approach? Or doesn't that matter at all?
Sorry for claiming to ask you one final question and then writing 5 question marks in one paragraph. I appreciate every tip you can share.
Really, "testing hypothesis" means testing whatever is quick enough to articulate a potential solution to an already-articulated design problem. That potential solution could be as small as a copy change or as big as a layout change. It depends on the design problem you're encountering.
The approach itself is qualitative, hence the "Lean." The objective is to get to a hypothetical solution as quickly as possible, qualify or disqualify it on its merits, and then refactor to stabilize as needed.
Thanks for your explanation. I definitely take a closer look at "testing hypotheses". I'll try to "google" the rest myself … thank you very much :)
The sad thing is most designers I work they usually think UX is wireframes, and they destroy all kind of useful information from the wireframes, like color usage, font sizes that were tested previously, etc.
While I disagree with some of the points about usability experts (not everybody in that world communicates through opaque reports, and getting good at usability testing is a distinct skill that takes time and effort to get right) - I whole heartedly agree with the advice that designers should get involved with usability testing their designs.
If nobody in your team is doing usability tests already, run and buy Steve Krug's "Rocket Surgery Made Easy". It'll take about an hour to read - and get you up and running with the basics very quickly.