Tree testing guide

Today we are going to explain how you can build a tree testing in three phases: define, develop and analyse.

Defining a tree testing

• The first step is to define the focus of your assessment (what are you going to focus on?) as well as its objective (what are you testing for?).
• Define the mode of testing. If you want to do online tree testing, you can use tools such as Treejack.
• Define how many and which people will participate in it (the recommended number is seven to nine, but the number may vary depending on the project), in order to be able to proceed with the recruitment. Those who participate must meet a very specific profile: the more affinity you have with the selection of people, the more relevant the information they provide will be.
• Define whether the test will be remunerated, as a thank you for the time and collaboration.
• Define the script for the sessions. Make sure that the contents (at least a consent form, a demographic questionnaire, a list of tasks — e.g. “book a trip with accommodation for two people, one of them with reduced mobility” — and a final questionnaire where open questions predominate) can be done in no more than one hour, to avoid participants getting tired.

If you want to learn more and go deeper into UX research techniques, don’t miss our Advanced Research Specialisation Programme (this training is in spanish):

Programa de Especialización en Research Avanzado

Develop a tree testing

For the smooth running of the test, stick to the script. However, these ideas will help it to flow more smoothly:

• Adopt an attitude that builds trust. This is important so that participants feel comfortable and act naturally. If the session is online, you can carry out some exercises to energise the sessions with users, which will help them feel more comfortable during the session.
• Set out the tasks one by one.
• Ensure that people can complete one task before moving on to another, regardless of whether the goal is achieved or not. The purpose of each task having a goal is to test a person’s mental maps, so that if the user does not reach the goals as planned, an iteration of the design will be needed to make the design more understandable. For example, if the task is “contact the Help Centre”, the objective will be considered as not met if the user fails to find or contact the Help Centre, and as met otherwise.
• Ask to be notified when they consider a task to be solved even if it is not: this is called a false success and is also relevant for the production of the report afterwards. In the example above, a false success is the user copying and pasting the general information email, and contacting you via that route, rather than looking for the Help Centre.
• Also ask them to verbalise their thoughts and feelings, for example when they say: “I am not sure how to proceed” or “this is irritating”. We can use probing questions to go deeper into the information the client is giving us.
• Look also at gestures. The body posture, the look of surprise or frustration at a task that is too difficult, the gestures of boredom: all of this is valuable feedback.
• Do not intervene in a task even if you want to “help”. This is very dangerous because it can lead to confirmation bias, which can invalidate the research results.
• Ask questions if you are unclear; better to ask too many questions than not to clarify what you need, but try to be neutral, non-judgmental and non-critical. For example, it is preferable to ask “Do you want to comment on task 2?” rather than “Why do you think it took you so long to do task 2?” The difference is subtle, but significant.

Analyse the results

This test will return quantitative data (e.g. demographic information from the questionnaire, and various rates of participants: the success rate per task, average time of task completion, the rate of errors and false successes) and qualitative data (e.g. annotations, notes on the possible emotional state of participants, answers to open-ended questions).

Here are three guidelines:

• Classify the observed problems according to their severity: critical (those that completely prevent the completion of a task), serious (can lead to people getting frustrated and abandoning the task) or minor. In addition, you can create an affinity map after obtaining all the tree testing results to organise the qualitative data.
• You need to produce a report to clarify and share the results. The important thing is that the report should include at least the objectives of the test and the number of participants, the methodology applied, results, conclusions and recommendations to implement in the information architecture.
• Remember to use clear, synthetic and agile writing. The list format is usually the most advisable.

In addition, this technique is not only easy to implement but also adaptable to all kinds of contexts: as mentioned above, it can be performed remotely, overcoming pandemic scenarios.

In short, it is a versatile tool whose success depends on a good approach and a good initial definition, and which can deliver very valuable information from real users when it comes to detecting failures in an information architecture.

This article is a translation of one published in our corporate website:

Guía para hacer tree testing | Torresburriel Estudio