Human-Computer Interaction (HCI) researchers argue user testing should be included in every step of the product development cycle. The same rules that apply to, say, pharmaceuticals should also apply to software development. Would you want health care workers to be using products that have never been user tested? Probably not — in fact, medical products wouldn’t be released if they failed usability testing. Why release software without including some basic user testing?
Now that you’re convinced, here’s your Usability Testing 101 lesson. The first thing to know: usability testing methods can be split into two groups, summative and formative.
Summative evaluation methods are classic models for user testing that typically involves 5-10 recruited users in an artificial lab setting with a moderator. They’re considered ‘summative’ because it comes at the end of project cycle or after the project has launched. The results can quantitatively show the usability of a system – such as the completion time, completion rate, and user satisfaction score from post-test questionnaires.
However, summative methods aren’t useful while the project is still in development. Without a working interactive prototype, users won’t be getting an accurate picture of how the system really works. It’s also expensive and time-consuming way to conduct user testing.
Instead, formative evaluation methods can be used to gather insights into the usability of current design and the results can feed into the design and development process. Here are three formative methods that were used for Fusion’s redesign to evaluate the design and gather feedback from the users.
Five second test
This is a simple test that can be done in person, or through online tool like Usability Hub. The premise: show a content page to a user for five seconds, and elicit reactions. It can be done through a set of questions if there are specific key objectives for the test. For instance:
- “What do you think the site is about?”
- “What is the most important information on the page?”
For the initial round of redesign testing, three users were recruited to evaluate how effective the designs were in communicating the branding of Fusion and how well the users understood what the site was about. Five second tests are useful to to gain insights into the first impressions of the site.
Longer user interviews aren’t necessarily part of this method, but can be used to further understand how the users liked the site and whether some design elements worked.
Guerrilla user testing / hallway testing
This is a casual testing method where users are approached rather than recruited – “Just ask someone in the hallway”. These tests are fast, and results can be fed into the design process right away. As with standard user testing, you can set tasks and see how users do the tasks. The talk aloud method, where users talk about what they’re doing and thinking, is useful in gathering rich and detailed data.
This method is useful when you want to test new designs and new ideas, and don’t have time or budget to set up a formal user testing process. Some design problems become immediately obvious when shown to people who weren’t involved in the project. We can theorize about whether users will see this UI element, or whether they’ll use it in the way we expect. Simple hallway testing can show if the theories are true.
Also considered a usability inspection method, heuristics evaluation is meant to catch any “big” usability problems before the design goes into production. The key difference between this and other methods is that it doesn’t involve users. Usability “experts” inspect the system against a set of heuristics and see where usability violations occur. Since heuristics are subjective and general, it is recommended to have more than one person to conduct the inspection.
Heuristics can cover a broad range of problems and identify areas that users can overlook. For example, “consistency and standards” of user interface is not something people usually look for. Jakob Nielsen, the originator of the heuristics evaluation method, has defined 10 major principles of interaction design.These 10 principles were initially developed for evaluating desktop applications. Only the heuristics that apply to the project should be chosen for evaluation.
After the evaluation, each heuristics violations can be rated in its severity and further fixes and recommendations will be made to the designs. With the redesign, some of the discovered heuristics violations were already being addressed as it developed.
Heuristics evaluation is meant to complement, rather than replace, user testing. The problems are predicted and are not from the point of view of target users. However, it can quickly yield evaluation data to improve the system while it’s still in design process.
What you should remember
Formative evaluations are designed to feed user feedback into the project as it develops and iterates. They are meant to detect usability problems. You don’t want to wait until the shipment of the product to learn that there is a usability problem.
User testing also does not have to be timely and costly. Steve Krug (author of Don’t Make Me Think) has even simpler form of hallway testing called cubicle tests, where you ask the person in the next cubicle whether they can understand your new ideas.
There are a number of other usability testing methods out there – it’s just matter of knowing exactly what you want to achieve with the testing and what you want to find out. And, step zero, committing time to do the testing.