Metrics That Matter : Measuring Product Success

As Product Managers, we can look at an infinite number of KPIs to tell us how we’re doing.  But what I often find is one of two scenarios: either there is no discipline around metrics OR you are looking at so many metrics that tracking them doesn’t really make a meaningful difference to your product strategy.

Last month, I was a speaker at a Product Tank meetup and the topic du jour was 'Measuring Product Success.'  It was a really cool event and my fellow speakers were brilliant.  We covered a gamut of topics ranging from Product Marketing to Pretotyping to IoT Product Management.

I got a great refresher on Startup Metrics for Pirates (AARRR) from Jordan Skole who is in Product Marketing at ActiveCampaign, some phenomenal stories about Airbnb from Kevin Marvinac who is now a Product Manager at ShipBob and most of all, a lot of stimulating conversation around Product Management & Entrepreneurship.

Similar to most presenters, I spent a lot of time telling stories around metrics.  I don't know that the stories themselves make sense in this post but I did want to share the key takeaways I talked about:

1. Focus on the customer and start with the end in mind: Product Managers have to constantly balance the unsexy work of improving core experiences and the sexy work of building features that are headline generating. In some rare cases, you will build headline generating features that are designed to improve core experiences but by and large, the bright shiny objects (ahem, order with emojis?!?!) rarely move the needle. So, my ask of you is simple: Keep the customer front of mind and really invest time in clarifying why you are doing something and who you are doing it for. Don’t get carried away in the joy of pushing the boundaries of innovation because you might find yourself spending a lot of money on something that won’t help your business. I recently spoke with a product manager from Amazon who told me they were required to write their future press release as a part of the product pitch.  I thought this was brilliant. You need to know what you are aiming for in order to know what to build and furthermore, what to measure to gauge success.

2. Choose two complimentary metrics to focus on: Depending on the stage of your company, you are typically looking at three buckets: acquisition, monetization and retention. When I first started at Walgreens, we focused our work around two key buckets: acquisition and monetization.  And then as we hit our stride, our focus shifted to reducing our customer acquisition cost (CAC) and maximizing our customer lifetime value (CLTV). In my current job, given our vision and goals, the customer experience we are designing is optimizing for conversion AND frequency. One of these in isolation will not guarantee the longevity of our business. If I optimized merely for conversion, I can get my customers to buy once but they end up not coming back. If I optimized just for frequency, I am probably leaving some money at the table through discounting etc. Optimizing for both ensures I am not making short term decisions that jeopardize our long term goals.

3. DO measure success: I have seen this play out in a lot of organizations. Business teams will submit requests to product.  Either the business team or the product team will put together an ROI analysis.  More often that not, this is a wildly optimistic view of how things will pan out.  Mainly designed to prove this project is worth doing.  Only in really disciplined teams have I seen the rigor that validates actuals vs. expected.  The more you do this, the more you will know the validity of your assumptions and the better your next ROI calculation will be.

My parting thoughts on this topic: In Product Management and in life, always take the time to learn from successes and failures.  And then reinvest these learnings to make better decisions.  If all you focus on is whether you were successful or not, but don't figure out the why, then you won't know where to make your next investment.

Here's to better decision making and better products!