Developing a Jira App: How to Design Usability Testing
Usability testing is a crucial step in designing any product or application. For our team developing Clear Path , designing the usability tests was a very important part of the process. Since our Jira app was a new application being developed from scratch, there was even more of a need to understand the user experience and improvements that could be made based on user feedback. Before conducting the tests, designing the usability test is equally important to ensure you are receiving the information that would help make your product better.
When first designing our usability test script, there was discussion about whether to use a usability test or user acceptance test (UAT). Although these types of tests are used interchangeably, they have differences that need to be considered. Being a new app, it was a question for our team on whether to do usability testing or user acceptance testing. So, what is the difference between these two types of tests?
User Acceptance Testing vs. Usability Testing
User Acceptance Testing is performed to ensure that a product or application meets certain users' standards and requirements. It is focused on validating that the product works and functions as intended, rather than examining the user experience. User Acceptance Testing is done before usability testing, and aims to answer the question of “Did the application solve the problems it intended to solve?” Designing a user acceptance test requires users of the target audience, and gathering opinions of users that have some sort of contextual knowledge of the problems being solved. UAT is conducted using scenario-based testing, where the user will have to accomplish many tasks to fulfill the scenario. User Acceptance Testing’s name does describe the main goal behind it: is this an application that the user will accept?
Usability testing is concerned with the ease of use for the user and the overall user experience. It is aimed to gather user feedback and identify any potential usability issues. Often there is a script with different “tasks” for a user to complete to identify potential pain points and areas of improvement for the product or application. Usability testing focuses on more specific, smaller tasks. Assuming the UX designer already completed user acceptance testing, usability testing follows up with the question “How well did the application solve the problems it intended to?” Usability testing measures the quality of the solution, even if user acceptance testing deemed it “accepted” by the user. Usability testing is generally the type of testing used for UX designers and researchers. For some applications, it can be assumed that the users already accepted/used an application. This is why it is important to test how well the application is performing and the experience of the users.
Our Experience
As you can tell, these types of tests are pretty similar to each other. Sometimes they are even used interchangeably or in tandem with each other. Our team had a couple of collaborative discussions on what we felt would be the best direction to go to understand the user experience the best. We decided to design a usability test, because we felt pretty good that Clear Path was solving common problems in Jira to differentiate it from other apps. We chose to focus on usability testing, because we wanted to measure effectiveness, efficiency, and user satisfaction in the ways we solved the problems. Because Clear Path is a SaaS product, we knew to achieve our goal of high user adoption and retention from our free trials we needed to ensure the product delighted our user base with its ease of use.
The Importance of Usability Testing
There are many reasons why usability testing matters and is so important for any project or application. UX designers are often reminded of the phrase “You are not the user”. This is an important phrase to remember, because even if a design makes perfect sense to the designer, that does not mean that the user will have a good experience with it. This is also the whole goal of User Experience design, making sure that your application provides the best user experience to the widest range of users. It is also important for early issue detection. Think of someone proofreading a paper you wrote and catching a grammar mistake, even though you did not notice it. The same is true for usability testing. Often, we are too close to the design and a fresh pair of eyes can help us find things we would have never noticed.
Designing a Usability Test
This leads us to the topic of how to go about designing a usability test. There are many steps and best practices that need to be implemented to design a successful usability test. UX designers will often first start by creating a “script” that entails test goals, user tasks, and any specific questions that are pertinent to the goals of the test. I am going to go over some of the steps when it comes to designing the usability test. This does not include conducting or post-test steps.
Planning the Usability Test
Planning the usability test is the most important step. The UX designer and any other team members involved in user testing want to make sure that the plan is detailed and in line with testing goals. For Clear Path, we primarily had a single developer , a product owner, and a UX designer. This made it easy to assemble the team and get feedback.
Determining the testing goals is a very important part of planning the usability test. This will help the team stay on track with the test, and make sure they are getting the most out of the usability test. Some example goals we used for Clear Path are: What are the pain points of the app, or what are the parts of the app that can be improved? Although these goals can be more general, our team is aware of what we might think a user would say the pain points are. With this, we can create tasks or questions to see if this is true for a user.
With planning also comes the logistics of the test. At this point, answering the who, what, when, where, why? questions. For Clear Path, we found internal employees that could complete the tests at their work locations.
Creating the Prototype and Deciding Tasks
For Clear Path, we were on an aggressive schedule, we chose to conduct usability testing after we developed the MVP. Because of this, we did not have to worry about creating a Figma prototype. However, you want to make sure that for any usability test there is a live or prototyped version of what you are testing so users can move through the test with more ease.
It is important to ask users general questions, such as “How do you feel about this page?” But it is important to have “tasks” for the user to complete that can help your team understand the user’s ease of use and usability issues within an application. It is important that these tasks are general enough, so they are not leading users, but also specific enough that the user knows where to go.
For Clear Path, we have tasks such as “Show me how you would go about visualizing a release.” It is a broad task but shows us how a user would truly go about this and can reveal information that you might not have been aware of. You can always help the user if they are not able to complete the task, and this will reveal that there might be usability issues for the common user.
When it comes to tasks, a big part of this is making sure the user does not feel “tested” on their own abilities or feel bad for not “successfully” completing a task. It is important to let the user know beforehand that we are testing the application, and not them. After the tasks, there are usually follow up questions that will wrap up the user’s overall experience and feelings about the application.
Finding the Right Users
For Clear Path, we knew our tool was for users that use Jira. Within our organization the primary users will be the product owners. However, other technology and business roles also use Jira, so it is important to get a versatile pool of users.
For planning a usability test, you might not be at the point yet to recruit the users. However, it is still important to have an idea of who the users could be and where you could find them. It is best to find at least 5 users to do usability testing. The number 5 is statistically significant, where most of your usability issues would come up using 5 different users.
Sometimes you might not be able to plan for the exact type of user, but generally it would be best to try and find people who have some contextual knowledge but also some with no background knowledge of your application. This will provide a holistic user pool and allow for more holistic results.
Deciding How to Translate Results
Even in just the designing part of usability testing, it is important to be aware of how the results will be translated. Testing can either be quantitative or qualitative. Quantitative testing is based on numbers and statistics. Qualitative testing is more abstract, but equally as important. Both methods are valid, but it is important to be aware of so you can get the most out of your results.
Important Things to Remember for Designing a Usability Test
Preparation Will Lead to a Successful Test
Preparation and planning for a usability test will make the test more successful. The goals and objectives need to be clear, and this will lead to an understanding of the user experience and allow for the overall success of your product.
Collaboration is Key
For any UX project, not just for testing, collaboration is key. It is important that the members of the team such as the product owner, UX team, and developers all can collaborate on the user testing. This will not only help with the general team's understanding of what is going on, but each person can offer fresh perspectives on the design of the usability test and potential tasks to be included.
The Main Goal is to Understand the User’s Experience
The main thing to remember when designing a usability test is that it is to better understand the user and the user’s experience. Being able to understand the users and get the most amount of feedback and experience with them will ultimately lead to a better product as well as a better user experience.
Contact Moser Consulting for more information.
