Metrics

Episode #8 of the course Intro to Product Management by Ellen Chisa

The Product Manager is continually looping through the three bubbles we showed at the beginning: UX, business, and technical challenges. We touched before on the beginning of a business—thinking about success and selling your product to obtain fit. But what else do you measure, and when?

There are several schools of thought on key performance indicators. One version is the idea that there is only one metric that matters. In this school of thought, a PM should pick the one metric they want to move and focus in on that. This tends to be a higher-level metric—for instance, Net Promoter Score (NPS). That’s what percentage of your customers are enthusiastically recommending your product.

Another school of thought is that the Product Manager should constantly be monitoring whatever is deemed a KPI—a Key Performance Indicator. These are the measurements that show if the product is succeeding or not from a variety of angles. For Highbrow, it might be that you made it to Day 8 of this class and you’re still reading! That would mean great retention. But if you’re the only user who ever signed up (you aren’t), that alone wouldn’t indicate success. Highbrow might opt to track both how many users sign up for a course and then how far they get in the course.

At the end of the day, there isn’t a huge difference between the one metric vs. small set of metrics schools of thought. Both are ways to make sure that you aren’t measuring so many metrics that it’s distracting. A PM can easily get swamped thinking about too many things or a metric that’s too complicated to discuss within the company. The key is to make sure that the metrics are understandable and useable. These metrics should always tie back to what you decided was important to the business back in Day 4.

Finally, Product Managers use these metrics to do experiments. The right time to do an experiment is when a PM has a hypothesis. If you’re changing the homepage copy, a likely hypothesis is that the new copy will lead to more sign-ups. Running the experiment allows you to validate or invalidate that theory. Setting up experiments and looking to learn is more valuable for a PM than combing through random data and trying to decipher value. The latter is more likely to be correlation than causation.

Recommended resource

Focus on the Metrics that Matter: Identifying Your Product’s Key Metrics and KPIs

Recommended book

“Lean Analytics: Use Data to Build a Better Startup Faster” by Alistair Croll and Benjamin Yoskovitz

Share with friends