September 13, 2019 User Testing 1 – What, How, Why?

How do you know if users are happy with your designs? Is it possible that they struggle with a feature? In this article I will talk about a great technique to evaluate the product of the design process – user testing. Some of you might already have some knowledge in this field, but those who learned design through the Topcoder model didn’t have a chance to get hands on experience…until now.

Why is user testing important?

As a designer, you might think you have adequate knowledge to create the best UI and UX without users. But you don’t. It is always the real user, preferably in his “natural habitat”, who can validate your ideas. Even if you think that you created a perfect flow, those users can still show you where there is friction. 

Types of user testing

You can run low fidelity, early-on tests based on paper prototypes. This technique can save a considerable amount of time and resources, as flaws can get caught early. The high fidelity tests can be run on partially functioning prototypes or even after launch on the real product. You can also classify testing as summative or formative. In summative testing we use a larger sample size of about 25 to 50 users. These tests are done on a high-fi surface remotely and we can gain mostly quantitative insights. In formative testing we need a smaller sample size of about 5 to 10 people. It can be done on low-fi prototypes, on site, and we can get quick, informal insights, mostly with both quantitative and qualitative results.

How to conduct user tests

1. Create a test plan: the first step of user testing includes recruiting users, defining the scope, identifying the objectives of the test and establishing metrics that you will use during the test. 
2. Facilitating the test: this includes observing the users, identifying the issues, identifying solutions and interviewing the users.
3. Analyse the test data: in this phase you assess the user behavior, assess their click paths, and identify problem areas
4. Create a test report: as a last step you would have to review recorded material (video, screen recording), identify design issues, identify best practices, and supply design recommendations. 

What do we get from user testing?

The quantitative metrics of user testing can be:

• Time on task
• Success or failure rates
• Confusion/task rates.

The qualitative insights can include:

• Stress responses
• Subjective satisfaction
• Perceived effort

Thinking out loud

Yes, we like Ed Sheeran, but this time it is not about his beautiful song. Thinking out loud is a technique that we should follow. Encourage test participants to talk about their feelings throughout the test session and basically narrate what they are doing, and why they are doing it. For instance, “I click on this button, as I think it will take me to the booking page”. The person who facilitates the test should take notes or listen to the recordings carefully to find the valuable insights from the speech of the test user.

What is user testing not good for?

User testing is not market research. For that you have numerous other tools to use. User testing aims to find usability flaws and help prioritise our next steps to correct mistakes.

In the next article I will show you some practical aspects of user testing involving two common tools. I hope you will join me for that story as well.