Creating a successful product requires more than just great functionality. To make the most out of your product, it is important to understand if your users are actually able to use the product effectively.
That's where usability testing comes in. It's a powerful tool that helps you evaluate your product's user experience and identify areas for improvement. However, if you're new to this concept, it might be difficult for you to decide on the questions to ask during usability testing.
In this article, we will explore examples of questions to ask in usability testing and share tips and tricks on how to ask them.
Read on to understand how you can gather the most useful results from your usability test.
A Brief Overview of Usability Tests
A usability test is a type of user research study that evaluates the ease of use and the effectiveness of the product, website, or system. You can conduct such tests during any stage of the design process–from early prototypes to finished products.
Let's say you are working on a website that curates playlists based on books. Once you're done with your prototype, you will want feedback on it from those close to you. You’d want to pay close attention to how they react to it, even recording it sometimes.
Doing the same for a set of possible users, but in a planned and systematic way is pretty much what we call a usability test. These tests usually involve observing the user as they use the product and collecting data on their interactions and feedback.
Usability tests help you understand where a product or website is confusing, unintuitive, or frustrating for users, before you build it out in code.
But before you jump into your usability test, you need to set up a discussion guide.
Why do you need a discussion guide?
Before you start an interview, it’s good practice to create an overview of how you want to run the test—an agenda. Such a plan for usability tests is called a discussion guide.
A discussion guide consists of a rough outline that you refer to during the test. It serves the purpose of guiding the conversations that you have with the user by:
- Ensuring that you’ve thought through what you want to test in advance
- Helping you collect data points on the same topics or tasks across users. You need to do this to see if there’s a pattern. Did only one user find an interaction confusing? Or was it a majority of them?
Here are a few tips for you to create and use an effective discussion guide:
1. Use the first few questions to warm up the conversation.
The purpose of the usability test is to get authentic feedback from the user. Creating a comfortable space for the user and their thoughts is a crucial prerequisite for this. Think of it this way–would you like it if someone you just met started quizzing you? Probably not.
Start with a few warm-up questions to learn about the user's background and context and give them a few minutes to get comfortable with you.
Examples of warm-up questions you can ask:
- Can you tell me about yourself? What is your role at your company?
- What's your educational background?
- How familiar are you with our awesome company? (we know it’s awesome but you can’t say that. Keep your questions neutral.)
2. Use clear and simple language.
You cannot have a productive conversation with your participant if they don’t understand you. To ensure that you’re on the same page, make sure that you don’t use technical terms when communicating. Keep the questions clear, concise, and easy to follow.
Some confusing questions and their clear alternatives:
Confusing question: "Did the product's user interface facilitate ease of navigation?"
Clear alternative: "Did you find the product easy to navigate and move around in?"
Confusing question: "What is your opinion of the product's design?"
Clear alternative: "What specific aspects of the product's design did you like/dislike?"
Confusing question: “What was your impression of the product's affordances?"
Clear alternative: "Did you find the product's features helpful and easy to use?"
3. Don’t limit yourself to the guide.
A discussion guide is an important tool for usability testing. But you need to remember that it is equally important to keep the conversation flexible. Keeping things conversational helps to get the best answers from participants (no one likes to feel like you’re reading off a script!).
We aren’t urging you to go rogue or off-topic. Just make sure you let the participant speak their mind. The participant will feel more open and heard, and give you better answers as well.
If you're looking for a brief guide on mobile app usability testing, check out "Mobile App Usability Testing: A Quick Guide”.
Types of Usability Testing Questions
The types of questions asked during a usability test can be divided into three categories based on when they are asked– pre-test, in-test and post-test questions.
1. Pre-Test Usability Testing Questions
Pre-test or screening questions are meant to inform you of your participant’s background. You may ask them their age, the gender they identify with, their educational qualifications, familiarity with your product/service, and any other background information based on the needs of your current research project.
Collecting such background information during usability testing can help identify patterns and correlations in your data. For example:
- How did users from different age-groups differ in their feedback of the product?
- Did people from smaller companies love how easy it was to collaborate?
- Were users from bigger companies more concerned about security?
The answers to these questions can be collected before you invite the participant over for a test or during the interview itself. Depending on what your research project demands, you can keep this stage interactive or go the async route with a pre-test screener survey.
Goals of pre-test questions:
- To establish rapport with participants and make them feel comfortable and at ease.
- To gather relevant demographic information about your participants, such as age, gender, and occupation.
- To assess participants' experience and familiarity with the product or similar products
Important Example Questions
- How much do you know about *your service*?
If you're still in the prototype stage, you can modify this question to ask about their awareness of a competitor's services instead.
- Have you used a similar service before?
If yes, follow-up questions are key! Learn all about those competitors
- Can you tell me about your job? What does your company do?
Knowing the participant's occupation and industry can provide valuable context during the testing session. For example, a user who works in the tech industry may have different expectations and preferences for software interfaces than someone who works in healthcare.
2. In-Test Usability Testing Questions
In-test usability testing questions are asked during the main course of the test. They are focused on the user’s interaction with your product or service.
The goal of this part of the test is to identify usability issues or bugs as the user tries the product live. You might discover that users were unable to navigate through the product on their own, or they found the Slack integration your team built very confusing.
In this phase, it’s not just important to listen to what a user says in this part of the test, but also observe where they seem frustrated or confused. This is how you’ll know what you need to change to make your product usable for your audience.
To prepare for this phase, you should have a prototype or a working version of your product ready, and a set of tasks for the users to perform during the test.
If you’re working with a prototype, it can be as simple as a 2-D printout, or as complex as a realistic Framer version of the product.
Make sure to ask users ‘Why’ they’re doing or not doing something in response to a task. This will help you gain insight into what the user was actually thinking (versus what you expected them to think). Observing their actions on your app/service are important at this stage.
Goals of in-test questions:
- To identify specific bottlenecks or usability issues that the user is encountering.
- To clarify or confirm assumptions made during the design process.
- To gain insight into the user's decision-making process and behavior while using the product/service.
Example of Usability Testing Tasks
First thing’s first—you’ll give your user a task. Here’s an example:
Please set up a new account on *product*. Please think out loud as you do so, sharing anything that you like, dislike, or find confusing. In case you have questions, feel free to ask them out loud but I may not answer them until the end of the test.
Important Example Questions
- What made you go to *a particular page*?
- Keep such questions open-ended and note their answers carefully.
- On a scale of 1-5, how did you like the design/service of *a particular section*?
- You may ask this while you're observing the user's experience with the product, right after the user completes exploring that section or feature.
- What did you like about *a particular section*?
- Similar questions can be asked about what they disliked and why.
- How useful did you find *a feature*?
- You gain a better understanding of which features are most important to users and prioritize development efforts accordingly.
- What made you not use *a feature*?
- This can give you an insight into the features that'll prove to be not needed, or need to be improvised.
3. Post-test Questions
Last, but not least are post-test questions. Gathering the participants’ feedback on the whole test or their overall views on the product can be carried out during this stage.
Goals of post-test questions:
- To gather feedback on the user's overall impression and experience with the product/service.
- To identify specific strengths and weaknesses of the product/service from the user's perspective.
- To gain insights into potential areas for improvement or future development.
Important Example Questions
- What feature did you find to be unclear/confusing?
- Possible problems or bugs with the service can get highlighted and prompt you to probe deeper.
- On a scale of 1-5, how did you like the design/conduction of *a particular section*?
- By using a standardized rating scale, you can easily compare and contrast participants' responses to the same question.
- Would you recommend this service to your friends and family?
- You can follow up this question by asking them the reason behind their answer.
- Is there anything that you found was new and innovative?
- Prompts participants to share their perceptions of new and innovative features or aspects of the product.
Question Bank for Usability Testing Interviews
Having a framework to lean on makes all the difference. Here's our handy list of must-have questions for your next usability testing interview. The questions are sorted by product type (B2B/B2C) and testing category (exploratory/feedback etc.)
Access the list of must-have usability testing questions here.
Conclusion
Usability testing can prove to be a great tool and can help improve your product a lot. The examples of usability test questions mentioned in the article above are just a starting point to a much better discussion guide that you can create of your own.
Analyzing the outcomes of the test is as important as conducting the test itself. Make sure that you invest in proper analysis of the outcomes and get better results for the betterment of your product. Working on negative feedback can improve your product a lot.
.svg)
