As a growing business devoted to the people who will use our products and services, user testing is paramount to our process. The main goal of user testing is to inform the design process from the perspective of the end user. User-centered design is focused on designing for real users, and user testing tells us who that person is, in what context they’ll use a product, and what goal they are looking to achieve. User testing is an will form an essential part of our design process – as it’s a fantastic way to understand how our user base interacts with our product.

UX researchers have developed many techniques over the years for testing and validating their ideas, and below are some of the ones we will utilise in our development process.

The different types of user testing below suit different types of goals. Ultimately, the best format of user testing depends entirely on what we’re looking to learn, and how much time we have available. So deciding which kind of testing best suit our needs in the moment in order to gather the most valuable feedback on the user experience of your product.

 


 

1. Usability Testing

Usability testing is the process of watching/tracking an actual user while they use your product to see if it’s in fact usable. Usability testing is the best way to understand how real users experience our product. It’s also flexible for collecting a range of information about users, and it’s easy to combine with other techniques. Usability testing is a cornerstone of UX practice. When it comes to usability testing, one of the most important decisions we’ll make is whether someone should moderate the session.

As a rule when in design & development we should be aiming to be testing weekly on a Friday with 5 users (ideally 1 of each persona) - more on the theory of this here and guide below: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Moderated Usability Testing

Moderated usability testing is practiced by those looking to obtain feedback from live users. During a moderated test, moderators are ‘live’ with test participants (either in person or remotely), facilitating them through tasks, answering their questions, and replying to their feedback in real time. Live communication with test participants is a strength of this type of testing, because nothing beats watching participants in real time, and being able to ask probing questions about what they are doing.

For moderated testing sessions, I’ve made our future guide to testing here.

When will we use it:

We will run these during the design phase, when we have  a design that hasn’t yet been fully developed. YWe will use these tests to find the potential issues of our working prototype. By watching participants reactions on our prototype, we can gather baseline data that can save us from spending a lot of design and development time on a product that’s difficult to use.

Things To Remember

When moderating you can help probe the participant to delve deeper, keep the participant on track, and help clarify any confusion. However, a very common mistake moderators make is to tell a participant what to do. There is a fine line between guiding and helping a user. So you need to find a balance to keep the participant on task, while not messing with their natural experience. When this balance is struck properly, even the most convoluted of tasks can provide rewarding feedback.

Unmoderated Remote Usability Testing (URUT)

Unmoderated remote usability testing, as the name implies, occurs remotely without a moderator. It offers quick, robust, and inexpensive user testing results.

This method is usually based on the use of usability testing tools that automatically gather participant feedback and record their behavior.

URUT has following benefits:

  • Participants complete tasks in their own environment without a moderator present, which leads to the product being used naturally.

  • URUT is conducted online much like a survey with pre-determined tasks, so it can be completed in the participant’s own time without requiring the hassle of coordinating schedules.

  • Unmoderated tests can also be run concurrently, allowing for a much greater volume. The turn-around time for unmoderated tests is often significantly faster than that of moderated tests. Data can be collected in as a little as a few hours depending on sample size and testing criteria.

  • The cost is usually quite low since you don’t need to pay for moderators or an equipment setup. You can get maximum value for minimum cost when tasks are written as clearly as possible.

When will we use it:

  • When we need to obtain a large sample in order to prove key findings from our initial moderated research.

  • When we have very specific questions about how people use a user interface for relatively simple task.

Things To Remember

  • URUT should not be used as a replacement for moderated usability testing. Instead, it’s best when you use it in conjunction with moderated testing.

  • The lack of a moderator means less control, less personal observation, and a higher risk of confusion. Thus, to run a test successfully you need to set clear expectations for participants—it’s crucial to ensure that tasks are clear and user-friendly. Check out our future guide for setting up URUT here.

  • Be mindful of how much time participants spend with test. Kyle Soucy suggests an unmoderated test should be 15–30 minutes in duration, and comprised of approximately 3–5 tasks, because the dropout rate tends to increase if a test takes longer.

 


 

2. Focus Groups

Focus groups are a tried and true method of communication between a researcher and users. In a focus group, you bring together 6 to 12 users to discuss issues. The group typically lasts about 2 hours and is run by a moderator who maintains the group’s focus.

When will we use it:

We will use focus groups not to assess design usability, but to discover what users want from our product—their personal thoughts and preferences. Generally we will use these before we start development of a part of our service to gather as much insight as to what they want there user needs are before we even start design & dev.

They can also be useful when looking at testing competitor products or services to pull out what users like and dislike and what works and what doesn’t quickly in one hit.

Things To Remember

The success of the a focus group depends on the participants as some can be easily swayed by other opinions so we will use these sparingly and when utilising making sure we get the right balance of personality.

 


 

3. Beta Testing

Beta testing allows you to roll out a near-complete product to individuals who are happy to try it and provide critical feedback. This testing method allows you to ask users questions after they have the new product, track their usage and have them file bug reports. Beta testing a good way to market our product and get constructive feedback in order to refine the design to improve the product.

When will we use:

We will use this testing when each iteration of our product is near complete and we  want to put it in the hands of the end users to gather feedback.

Things To Remember

Sufficient testing should be carried out before releasing a product to our users as we don’t want our users to find and report bugs, we simply want their feedback on product features and usability.

 


 

4. A/B Testing

An A/B test is ideal as the appropriate testing method when we  are struggling to choose between two competing elements. This testing method consists of showing one of two versions randomly to an equal number of users, and then reviewing analytics on which version accomplished a specific goal more effectively.

When will we use it:

A/B testing is good when trying to detect small differences in designs. This testing is particularly valuable when comparing a revised screen to an older version or if we have split feedback from user testing on a new design. If there is no clear way winner from the testing we will release 2 version to small segments of our audience and use the data to decide.

Things To Remember

With A/B testing you only find the best option from among the available variations. These variations should be selected very carefully. If the variations are only based on internal experience and opinion, the testing won’t find the optimal design.

 


 

5. Surveys

Questionnaires and surveys are an easy way to gather a large amount of information about users, with minimal time invested. The right questions can uncover customer needs, desires, and pains.

When will we use it:

Surveys will help us accumulate quantitative data about overall user satisfaction or collect quantitative data to support qualitative research findings. We should always aim  to use them in support of qualitative data sets.

References & Resources:
https://library.gv.com/gv-guide-to-research-847cfb08fcef