UX Researchers are expected to multi-task, solve problems using different gadgets and tools and save the company. Heard of Batman, anyone?
Automating some arduous tasks can help you save Gotham faster.
Launched in 2007, UserTesting.com is a qualitative user research tool that allows businesses of all sizes to test their website/app/prototype and get feedback from users on video calls. UserTesting.com gives end-to-end remote user research facilities from recruitment to research analysis which enables businesses to save money and time.
Companies are increasingly understanding the importance of user testing with a study by Jeff Sauro MeasuringU revealing that testing products on 5 users can show 85% of problems in an interface. By using tools like UserTesting.com, companies can test continuously as much of the UX Researchers' tedious tasks get automated.
In this article, we'll review UserTesting.com and find out if it is ideal to use for your business by diving deep into its uses, how it functions in moderated and unmoderated testing, and the pros and cons of using this tool.
Pssst...Curious to learn about other tools? Check out our list of the 20 Best UX Research Tools
1. What tasks can UserTesting.com help you with?
Recruitment of Research Participants
UserTesting.com's suite of services starts from recruitment itself. Companies can select from UserTesting’s large pool of 1.5 million users from all parts of the world. UX Research participants are vetted through a test, which ensures that only high-quality participants make it to the platform. The quality of the participants is also continuously monitored and maintained through the ratings that other researchers have given to them.
After onboarding participants, they have to qualify on the screener questions that UX Researchers have created. UX Researchers have the option to save the screener questions for another study.
UserTesting.com then matches the profile of such participants to the company's requirements. It ensures that security and privacy are maintained by both parties.
Companies are also given the flexibility to recruit participants on their own. Lastly, UserTesting.com handles the issuing of payments to participants which happens over PayPal on the platform.
While it streamlines the recruitment process, the cost of recruitment is high. This can be a barrier for smaller businesses. Although the tool has a diverse user base to choose from, it only covers some of the world (covers around 40 countries). This can be a problem for companies with specific geographical requirements.
Conducting Tests
UserTesting.com offers a wide range of testing methods such as live interviews, usability testing, A/B testing, card sorting, tree testing, and clickstream tracking. UX Researchers can build their tests from scratch or use templates.
The first part would be screening questions which help UserTesting.com to identify participants according to parameters such as demographics, language, etc. UX Researchers need to select the type of test to be conducted- moderated or unmoderated. The videos of participants are automatically recorded while testing.
Usertesting.com is available only in nine languages which can pose a challenge for UX Researchers testing in other languages, right from recruitment to testing and analysis. While testing, user researchers will have limited contextual information about the participants such as their past experiences and personal preferences.
Also, participants can get influenced by the compensation they receive, thus leading to potential bias in testing and insights generated. We’ve heard of many cases where researchers have felt like their participants were “professional testers” on UserTesting.com—simply there to get paid.
Analysis
UserTesting.com partially supports research analysis. Here are some of their prominent offerings for analysis:
- UserTesting.com automates transcription generation for you. Be warned: the quality of transcripts is mediocre and some basic plans may not even include this feature!
- UX Researchers can also trace which web pages the participants interacted with the most by using their click maps feature for unmoderated tests
- Easy video clip creation for
Teams that use UserTesting.com typically end up in other tools like Miro and excel to analyze their data.
If you’re looking for high quality transcripts, auto-generated notes for calls, and analysis in one place, try using a product like Looppanel.
2. How does UserTesting.com hold up during moderated testing?
UserTesting.com offers the option of moderated testing where UX Researchers can interview or test the products with the participants in real time.
This allows for follow-up questions and in-depth analysis. This qualitative user research tool gives options for video recording, screen-sharing, and note-taking. Besides, a researcher from UserTesting.com will be available throughout the session to give technical assistance. This helps address technical glitches such as Internet connectivity, poor audio or video quality which can otherwise cause problems. After the session, the option to summarise the session is available.
Earlier, UserTesting.com did not automatically transcribe the moderated session. But now, UX Researchers can get transcripts of the videos and can directly copy and paste quotes for reports. The option to create video clips and share them with stakeholders is also available by clicking on that section of the transcript. However, the transcript won't be 100% accurate. For higher quality transcripts, try a product like Looppanel.
Currently, UserTesting.com supports only Zoom and GoToMeeting video conferencing. While UX researchers can use other platforms, UserTesting.com won't be able to help if any technical issues come up. This includes any backup recording.
3. How does UserTesting.com hold up during unmoderated testing?
UserTesting.com is a useful qualitative research tool that allows unmoderated testing where participants are asked to perform tasks on a website or app while the session is getting recorded.
It offers different types of tests such as A/B tests, click tests, tree testing, card sorting, and many others. The setup process for the test is quick and easy. UX researchers can type in their questions and instructions for the participants like thinking out loud while using the product. UX researchers can also add follow-up questions. After publishing the test, the participants can try it out. Just like the moderated test, the video is recorded. Once participants have completed the test, results will be out with an option to summarise the session.
UserTesting.com has a limited way to customize tasks. They have a standard set of tasks that is not entirely customizable according to the company's needs. This includes certain features that cannot be tested on the platform. This may prevent UX Researchers to try out certain tasks over the participants. Similarly, companies cannot fully control the order in which participants can perform tasks. All these factors can reveal basic usability problems, but may not be able to show complex issues of the website/app. Besides, they can impact the overall accuracy of the feedback.
If you’re looking for alternatives for unmoderated testing at a lower cost, products like Maze.co and UseBerry.
What are the major pros and cons of UserTesting.com?
A quick recap of the above, here are the main pros and cons of UserTesting.com:
Pros:
1. Quick set-up and easy-to-use
2. Recorded video clips with tagging and sentiment analysis that can be shared
3. Large pool of participants for recruitment
4. Free resources to guide UX Researchers
5. Constant technical support
6. Fast Results
Cons:
1. Customers cannot customize their study
2. Difficulty in exporting multiple tests
3. Lower accuracy in transcription
4. Harder to find participants with very specific requirements (especially in B2B industry/positions of seniority)
5. Not much flexibility in replacing a participant quickly whenever any issues come up
6. Participants may be biased “professional testers” who do not represent actual users
7. Cannot edit a test after launch
8. Very high cost, especially for smaller companies
How fair is their pricing model?
The biggest problem is their pricing model which is way too expensive. The lowest pricing model starts from $20k annually. Prices are customizable according to the business needs.
They offer three types of pricing model which covers the following features (as of April, 2023):
1. Essentials
- Creating Tests and using templates
- Filtering participants via demographics and screener questions
- Access to transcripts, dashboard, and metrics
- Connections with Jira, Slack, and Trello
2. Advanced Plan
(All the features in Essentials +)
- Planning a test and managing the audience
- Live Tests
- Analysis in the form of sentiment analysis, click maps and interactive path flows
- Unlimited workspaces
3. Ultimate Plan
(All the features in Advanced +)
- More options for tests such as card sorting and tree testing
- Managing custom audience
- Testing with Prototypes
- Tagging in analysis
- Approval flow
To avail of full benefits and all features, companies will have to go for the third pricing model, the ultimate plan, which is the most expensive. This may be too expensive for mid-sized and smaller companies that cannot afford tens of thousands of dollars a year.
What if you don’t have the budget of a FAANG researcher?
UserTesting.com has many benefits and can help in end-to-end research projects, but the pricing model is one of the biggest problems. Their cheapest plans don't include all features and upgrading can be difficult for many small and mid-sized companies.
If you're looking for a qualitative research tool that can perform similar functions at a way lower price, check out Looppanel. Looppanel streamlines the entire user research project at just $30 per month.
Don't trust us? Try out Looppanel for free for 15 days!
.svg)
