written by Jan Völker UX Architect
How to uncover all your usability problems with just five testers
How to uncover all your usability problems with just five testers
November 02, 2018 | Categories: Behind the scenes, Stories, Cases & Examples
About the author Jan Völker Jan Völker UX Architect

Usability Testing costs a small fortune and afterwards you know just as much as before. Not quite! We’ll show you how to detect usability problems with a handful of people – in the twinkling of an eye.

The Usability Test Dinner

On October 16, AOE invited companies and volunteers to a  Usability Test Dinner in Wiesbaden. The Open Source event takes place regularly in more than 20 cities and makes it possible to test the usability of products you have brought with you using the Thinking Aloud process. In addition to pizza and cool beverages, the products are evaluated by the testers according to the speed dating principle. The testers rotate every twelve minutes and are given a new test task. At the end of the evening, each product was tested by six test persons.

Why it’s worthwhile to pay attention to usability

You couldn’t find the headphones you were looking for in the online shop? Did you need to re-enter everything again when entering your credit card details for the third time? Were you unable to find the “purchase” button? In such moments, your users lose sympathy and trust regarding your product – and you lose money. It’s worth it to check and maintain the usability of your products. And it isn’t really that difficult.

Sarah Kay
The test dinner gave us valuable insights, which we then presented to our client. The test results support our recommendations, facilitate communication with our customers and improve usability.
Sarah Kay
UX Architect
AOE

The Method: Thinking Aloud

With “Thinking Aloud” we would like to introduce you to the superstar of usability test methods. The implementation is uncomplicated, requires little preparation and delivers meaningful results as a qualitative test method.

What do you test?

Prepare a task for your test persons that puts relevant elements of your product to the test. Here are a few examples:

  • “Purchase the most expensive red wine.” 
  • “Find out the diameter of the large Pizza Funghi.” 
  • “Rent a Tesla for the weekend.” 

Choose a task that is as manageable as possible and can be mastered by a test person within a reasonable period of time.

If you do not have a finished product, prototypes of any kind are just as appropriate. With sketched wireframes or digital click-dummies you can find out just as well whether an idea or a concept basically works.

Bernhard Behrendt
During the test dinner we noticed that our discount promotions unsettle the user more than they encourage him: The more expensive price had to be chosen to get the discount. We will now display the discounted price in a more eye-catching way, and show the regular price crossed out.
Bernhard Behrendt
Software Developer
AOE

Please think out loud

Pay attention to where the stumbling blocks are. Where is something expected? Where is something different expected? Statements such as: “Where is the back button here?”, “Here I would have expected to find a search field”, “What just happened”, give you valuable insights into the minds of your users. And you should take a closer look at an issue no later than when the third tester is stuck in the same place as his predecessors.

Users and their expectations

Pay attention to where the stumbling blocks are. Where is something expected? Where is something different expected? Statements such as: “Where is the back button here?”, “Here I would have expected to find a search field”, “What just happened”, give you valuable insights into the minds of your users. And you should take a closer look at an issue no later than when the third tester is stuck in the same place as his predecessors.


Usability Test Dinner – Pizza, Beer and Prototypes

Here’s a video with a few impressions of the Usability Test Dinner at AOE in Wiesbaden. Six test persons test all products – but networking, fun and pizza don’t come too short!

Anne Krones
During the test dinner, we noticed that our approach was not understood by the users. To arrive at this realization before writing a single line of code saved us and the customer a lot of time and money. In the spirit of “Fail fast, fail often, fail cheap”
Anne Krones
Product Owner
AOE

Are five people really enough?

If you are wondering whether you can really find all usability problems with just five people, we can reassure you: Based on empirical research by the Nielsen Group, approximately 85 percent of all usability problems can be found with just five testers. With ten people, this rate is increased to about 98 percent.

You don’t have to test with ten people at once to improve your results. Simply test multiple times – the method is uncomplicated. Use colleagues, friends, relatives and passers-by. Test different parts of your product. Repeat the test with each new version. Digital products usually evolve naturally – once you’ve identified a problem, try to fix it and do another Thinking Aloud session.

The Nielsen Group also recommends conducting small-scale testing as often as possible and even goes so far as to call large-scale testing a waste of resources. For the sake of completeness, Thinking Aloud is not a panacea that makes all other methods superfluous, or that highlights all aspects of usability – but if you haven’t tested usability yet, use Thinking Aloud. Why not try it right now with the colleague sitting next to you?