By Raj Kissy, Head of Division – Leeds
From my personal experience, (pre Agile and the Digital Revolution) Product Metrics in Project Management have been around for quite some time, but were being identified as something like Project Benefits or Benefits Realisation Metrics. The Benefits Realisation process would identify metrics, benefits or value a project may have to the organisation. The only issue was that these were specifically created and centred around the project itself (which obviously were valuable at the time).
Since the rise of Lean Start-Up (coupled with the internet and Digital tools), the ability to change (‘Pivot’ in the Lean Start-Up language) has become such an important factor for all organisations.
We have seen companies like Netflix rise and Blockbuster fall. Had Blockbuster jumped on the streaming based subscription services sooner, then we may never have seen Netflix but their inability to change quickly meant they lost business and ultimately (and unfortunately) went bust.
Obviously there is a lot more to the story of Netflix vs. Blockbuster, but the principle of changing direction (think of a sports car over a HGV lorry) is something any Product will want to be able to do. For example, you would want to know your Product is haemorrhaging customers within 1 day over 1 month, as you would be able to perform root cause analysis within 1 day and fix the problem almost immediately.
The flipside of Product Metrics is knowing the right metric to evaluate/monitor, as many Product metrics could be seen as vanity metrics, e.g. ‘Think about a marketing landing page for an eBook download. Measuring pageviews doesn’t allow you to make a business decision, but measuring download rate might inspire you to test out different on-page wording, call to action buttons or styles of form submission’ (Ref: https://www.tableau.com/learn/articles/vanity-metrics).
Therefore, product metrics should ultimately line up with the vision, mission or strategy of the business as set out by the C-Level team, to ensure you have full alignment of every Product Metric, and that it’s linked directly to a business outcome.
There are many tools and techniques you can use to ensure your Product Metric lines up with a desired business outcome, from VMOST (link provided below) to OKRs (link provided below). I have used many of them, but the purpose of this blog is not to review each of the techniques, instead to give you collateral to make your own informed decision.
OKRs have recently become heavily used since Google took advantage of this technique to measure their products and, as we know, they have grown from strength to strength.
The North Star framework is an interesting one I haven’t used, but having read about this technique, the principles seem to be the same (i.e. keeping the business vision when setting out the success measures/metrics for your products).
Summary
I’ve pulled together my own list of essential points to consider when reviewing your product metrics in Product Management. These are:
- Ensure you have true alignment to your business outcomes (Mission/Goal/Vision etc)
- Ensure you have the correct monitoring in place before you deploy a feature or new product. This will ensure you spot good or bad trends when you go-live
- Have a good reporting mechanism for the metrics. This will keep your stakeholders and teams informed of the metrics
- Try to stay away from Vanity metrics. They could give you false positives, and whilst they may potentially make you look good on the outside, they may not add any real business value.
- Don’t just think about the £££, as sometimes focusing on customer experience could lead to extra £££
- Don’t be the Super Hero – work with your development team to create the metrics, as they will have a sense of empowerment and ownership for the product.
Product Metrics articles
For further reading too, here are a selection of related article that I’ve found useful around metrics too:
- https://www.producttalk.org/2020/05/product-outcomes/
- https://amplitude.com/north-star/about-the-north-star-framework
- https://www.scaledagileframework.com/wsjf/
- https://felipecastro.com/en/okr/what-is-okr/
- https://www.toolshero.com/strategy/vmost-analysis/
- https://amplitude.com/blog/n-day-retention-for-mobile-games
