5 of the most common mistakes in product analysis
As product data analytics is just one tool for UX Researchers, it is quite likely that at some point in our professional careers we have made mistakes.
We are humans. In fact, our discipline, UX research, allows us to identify and fix those errors through iteration.
Thanks to this experience, at Torresburriel Estudio we have identified 5 common mistakes and ways to solve them:
- Not measuring actual engagement
- Forgetting about qualitative data
- Paying too much attention to noise
- Looking only at declining data
- Getting flooded in data
We are going to reel off all these points, with the aim of creating a product data analysis strategy that serves both UX researchers in their tasks and product managers when making decisions.
Not measuring actual engagement
When we work on a digital product, there are many metrics at our fingertips. New users or downloads of the product are just two which get a lot of attention in the analysis of product data.
However, these two metrics are essentially used to determine the adoption rate, but not the actual product use. If we want to measure engagement, we must look at other metrics that refer to the actual use of our product.
Which are those metrics? The ones about frequency (number of pages per session) or actions (who moves forward on a conversion funnel) will be much more useful to measure those interactions within the product.
Forgetting about qualitative data
If choosing the right KPIs is important, choosing the best sources of information is also critical.
Talking directly to your users is essential and that feedback will help you to know what product strengths and weaknesses are. Therefore, it is important to keep communication channels open: support, surveys or in-depth interviews will help you understand how your users interact with your product.
Is it more expensive than building a dashboard? Of course. But the data we can get from those conversations will help understand the real whys of everything we see on those metric dashboards.
Paying too much attention to noise
Although feedback is important, it is no less important to get its best value. There are different strategies to be able to use it and take advantage of the best of everything that those who use our product tell us.
The first thing to keep in mind is that we, UX researchers, are not users of our own product; but also, those enthusiastic users who share with us their opinions and submit feature requests may not represent our ideal audience.
All that information is still valuable, obviously, but it can be affected by user biases (they come from another application, it is not the main use of the product). For this reason, in our research strategy we must activate users who do not share their opinions, but represent the standard personas.
Looking only at declining data
When a critical business metric goes down, it’s probably too late. In fact, we can anticipate a potential issue if we put metrics together.
A clear example is when churn rate increases. In addition to external factors such as new competitors, we can anticipate those cancellations through metrics such as logins or usage rates of some features. If users access less to the product or we see an unexpected increase in data export requests, those indicators can anticipate a possible increase in the churn rate.
Hence, it is so important to look at the metrics that allow us to properly understand usage rates, since on many occasions they will help us to anticipate adverse circumstances.
Getting flooded in data
Adding more and more metrics will not help us understand user behavior. In fact, looking at too much data can lead us not to be able to make decisions or reach the wrong conclusions.
As UX researchers we must always focus on what we want to measure and design a research strategy to access data that allow us to understand behaviors and get appropriate conclusions.
This also affects product managers, since measuring too many parameters can cause them to lose focus on what is really important, even affecting the product roadmap.
These five errors can be easily corrected: proper planning of both research and product strategies, we will be able to analyze the data we need so that all our data analysis is correct and, with it, reach actionable conclusions for a more successful product.
This article is a translation of the following one published on our corporate website:
