How to prepare a Think Aloud Test
The procedure for the Think Aloud Test is divided into three phases: preparation, conduct and analysis and presentation of the results.
In this article we will focus on the first phase, the preparation of the test, breaking down each point.
The test preparation phase should not be underestimated at all, because the success of the test depends on good preparation. Within the preparation phase, the following activities can be distinguished:
- Determination of the general objectives of the tests
- Definition of usage scenarios and tasks
- Definition of the user profile and user selection
- Definition of usability objectives
- Preparation of materials and logistics
Determination of the general objectives of the tests
The first thing to do is to decide what the overall objectives of the test are: it is most likely not possible to test the entire website, app or digital product because we do not have much time at our disposal. Also, it is likely that not all areas of our product are equally complex and need to be tested. So we have to locate the most critical and potentially problematic areas.
Definition of usage scenarios and tasks
The definition of use scenarios and tasks is closely connected to the decisions made earlier. Once the critical areas have been located, we have to build one or more realistic and logically coherent usage scenarios. Each use scenario may comprise one or more tasks to be executed. A typical example of a usage scenario for an e-commerce that sells books could be:
- Task 1: Register
- Task 2: Search for a certain book in the catalog
- Task 3: Buy online
- Task 3: Check the status of the order
- Task 4: Close the order
Much attention has to be paid to the construction of scenarios and tasks that do not implicitly provide information that helps the user to carry out the task. Furthermore, the description of tasks has to be as aseptic as possible, so that users know precisely what they have to do, but can freely choose how to finish their task.
It would be good for the neutral observer to try to execute the tasks he assigns himself or have them executed by another tester.
The definition of the user profile depends on the analysis of the requirements of the end users of the site, which should have been done earlier. If this analysis is missing, it will be necessary to rely on the marketing and communication studies of the project. If the latter are also missing, there is a problem at the source and the Test Think Aloud becomes meaningless.
Once the profile has been defined, the users have to be selected. We have to decide how many users to test. Fortunately, here we can count on the help of two masters: Jacob Nielsen and Tom Landauer. In a research study, the results of which were published in March 2000, they demonstrated (and argued mathematically) that by testing with 5 users, 85% of the project’s usability problems can be found. Nielsen himself advises not to exceed this number of users, so that the execution of the test and the elaboration of the data found is not too costly. He also suggests a criterion for deciding the number of users to involve in a usability test:
- 5 users, if the product is intended for one type of user only
- 3–4 users per typology, if the product is intended for two types of users
- 3 users per typology, if there are more than two types of users
On the other hand, Steve Krug, in his famous book “Don’t make me think”, believes that it is better never to exceed 3–4 users, so as not to alter the nature of the test, which has to be quick. We can therefore say that the ideal number of users is between three and five.
It is important to remember that we are talking about an iterative process: after having corrected the usability problems that arose in the first Think Aloud Test, we have to test to find out if our solutions have achieved the objective. Moreover, in the redesign of our interface we might have introduced other errors. Therefore, the cheapest solution is not a Think Aloud Test with 15 users… It is better to do three tests with 5 users each.
Definition of the user profile and user selection
How important is it to select users who exactly match the profile of our user persona? It is important, yes, but how important? Let’s say our user persona is made up of pharmacists. Do we have to select five pharmacists? First of all: common sense. If we can’t find the users that match our user persona or we can’t because we don’t have the budget, we will look among acquaintances for those who match the profile of the user persona. Without forgetting that it is important that they are not people who have had a relationship with our project.
As we always say, researching is always better than not researching.
Definition of usability objectives
It is time to define the usability objectives of our test, or rather, the metrics to be adopted. Remember that the main objective of Thinking Aloud is to identify the usability problems of the interface, but we could be interested in other usability metrics, such as effectiveness, which can be measured through the success rate, in other words, the percentage of tasks carried out successfully.
We can also measure efficiency through the time taken to perform each task. But this is a metric that can be affected by the double request made to the user: to do the task and to explain it. Verbalising thoughts influences the time in which the task is carried out but, contrary to what one might think, it does not slow down execution times. It has been shown that in some cases users who verbalised the task performed it faster than those who performed it silently. Therefore, efficiency is a purely indicative value but not a determining factor.
Preparation of materials and logistics
The preparation of the tests is concluded with the phase of preparation of materials and logistics. As far as logistics are concerned, it is enough to have a room (you don’t need a UX Lab as for other tests), a computer connected to the network and a table with two chairs (one for the user and one for the neutral observer). Obviously, depending on your budget, you can implement more tools such as a camera to record or an eye tracking device.
In terms of materials, two things should emerge from this phase: the test plan and the script.
The plan is a summarised document that has to contain an overview of all the phases of the Think Aloud Test: objectives, methodology, list of tasks, user profile, test execution scheme, structure of the report in which the results will be presented (read our Tips for writing UX research reports).
It is also useful to prepare a test script, which is a real sequential outline of the events that have to follow one after the other during the test. This script can be very useful to the neutral observer, and should provide for this:
- Introduction of the neutral observer and description of his role as a neutral observer
- Explanation of the objectives of the test
- Explanation of the tasks
- Explanation of the methodology, with an invitation to the user to speak out loud during the test
- Make the user feel comfortable
- Make the user sign confidentiality agreements
- Filling in questionnaires, if necessary.
