Formative vs. summative evaluations
The most important thing in user experience is iteration and evaluation. There are two types of evaluation, formative and summative, and depending on where you are in the project design process determines the type of evaluation you should use.
Formative evaluation is focused on determining what parts of the design are working correctly or not, and will try to find a reason in case it doesn’t work as expected. These evaluations are conducted during the redesign and provide feedback for interface improvements.
If we were prototyping a solution for a new app redesign, we would test it with users to see how usable it is. This study could identify various problems in the prototype and then we could fix them in a new prototype. This test is an example of formative evaluation, as it helps designers identify what needs to be changed in the early stages of the project to improve the interface.
Formative interface evaluations are carried out with testing and changes to the product (usually several times), until the redesign process is completed or while creating a new product. Formative evaluations aim to improve the design in the right way.
Summative evaluations describe how well a design performs, frequently compared to a benchmark that can be a previous design or a competitor. Opposite to formative evaluations, which are intended to inform the design process, summative evaluations include getting the full picture and evaluating the entire experience of a final product. Summative evaluations happen less frequently than formative evaluations, typically just before or after a redesign.
Using the app example, if we have launched a new app, we can test how it performs compared to the previous version. We can calculate the time per task and the success rate for the main functions of the app. We can also compare them with the data obtained in the previous version of the app and see if there are any improvements. This research is a summative evaluation, as it assess a launched product with the objective of measuring performance over time.
Research methods for formative and summative evaluations
A common misconception is that summative evaluation is the same as quantitative research, and formative evaluation the same as qualitative research. Both formative and summative evaluations can be either qualitative or quantitative.
Summative evaluations are usually quantitative, but can also be qualitative studies. One case might be that you want to compare how your product performs and is viewed compared to a competitor’s product. To achieve this you could pay for a heuristic evaluation of both your product and a competitor’s product by an UX expert.
Moreover, formative evaluations are not always qualitative. For example, a quantitative study can be carried out to find out the percentage of people who scroll down a certain page and to be aware that it is enough to get results.
Formative and summative evaluations are aimed at different research objectives. Formative evaluations are used to improve the design so that the final product is perfectly optimised for user experience. They are a normal and natural phase of an iterative user-centered design process. Summative evaluations are basic in assessing the global usability of a product and are essential in mapping its usability over time and in comparison to its competitors.
This is a translation of the following article published on our corporate website:
