The Dark Side of Product KPI

Why do we base product KPI on data to begin with?  Product KPI (key performance indicator) is defined so we can measure how well the new feature works or resolve an A/B testing. That’s why it is only natural that when we come to set the KPI for a new product or feature, we start by looking at the data. This methodology fits nicely with the "scientific" and lean approach to product development: We base decisions on data and measurable KPIs rather than "soft," qualitative guess work. See also: 10 Trends on Big Data, Advanced Analytics   The limitations of working only with data But confining ourselves to data has its limitations. For starters, data is not immune to biases. We tend to interpret data to confirm our assumptions. But even if we reduce the bias risks, we can only look at the data we have. So if we already accumulated a lot of data, it still does not include behaviors and use cases that occur beyond our data, in the dark side of the data. Defining product KPI based on existing data is like looking for the solution under spotlight. That’s fine if the solution is there, but what if what we need to improve in our product market fit, or hack in our growth challenge, is hiding in the shadows? The importance of articulating a product strategy  Product life-cycle often starts with use cases. We ask ourselves what do our users do and how can we solve their problems or improve their experiences. That helps us to define the product KPI. But there is another important step in between. That is articulating product strategy. That is to say: given certain use cases, what do we want our product to achieve. Is it simply to help existing users do something faster? Or in a more sharing manner so we can hack viral growth? Or, are we looking for the product to serve a more business oriented goal of up-selling new features? Or, even help attract new types of users? While these questions may not change the basic use cases, nor our deep dive into product flow, these questions could be critical in defining the product KPI and eventually measuring product success and ROI. See also: Big Data? How About Quality Data?   Using product strategy to define product KPI A well-articulated product strategy can influence our prioritization along the product life-cycle and make MVP decisions easier. We still want to test our value and growth assumptions first, but without a well-articulated product strategy we can find ourselves arguing about the definition of value. There are many ways to generate value in a certain use cases. Setting the product KPI based on product strategy means we are prioritizing the value proposition according to our preferred target segment and our business goals. That’s why when we define product KPI it’s not enough to look at data and use cases. We must define product KPI in the context of our overall business strategy and its derivative product strategy.