Blog refresh: How to actually measure product engagement #14045
Description
Summary
This is the outline for a significant refresh of an old blog, How to measure product engagement. The current version doesn't explain product engagement very well and recommends an outdated framework. This refresh has a more structured and opinionated take on how to measure product engagement and takes a lot of inspiration from the Getting Hogpilled doc.
The goal of this blog refresh is improved SEO and general writing practice :)
Brief outline:
- First, we explain what product engagement is conceptually.
- Next, we list and explain some common metrics/ways people use to measure it. (stickiness, retention, etc.)
- Then, we dispel the myth of Product Engagement Scores (PES), which is a composite metric described by some of the top SEO hits. We explain why PES is extremely reductive and rarely useful.
- Finally, we recommend that startups optimize for a North Star metric instead of (or at least, in addition to) various traditional product engagement metrics. We also note the value of qualitative data early on.
- The summary will include a note on how all of these different concepts are semantically related to each other + final thoughts/takeaways
Headline options
- How to actually measure product engagement
- What no one tells you about measuring product engagement
- Product engagement metrics explained
- Stop using product engagement scores (here's what to do instead)
- The right way to measure product engagement
Outline
What is product engagement?
Product engagement is, at its simplest, how people interact with your product. This includes how frequently they use it, what they do in it, how deeply they engage with features, whether they keep coming back, and more.
We measure product engagement as one of many ways to understand whether users are getting value out of our product. For startups, it's especially useful to think about high product engagement as a signal of product-market fit.
Common ways to measure product engagement
To accurately measure product engagement, you need to choose the right metrics in the right context. It's not always about measuring how much people are using your product, for example. Someone spending 3 hours in your tax software because they're confused by your UX is not the same as someone using a meditation app every night for only 2 minutes a day.
Another consideration is that different product types and industries will have very different product engagement metrics (ex: B2B collaboration tool vs consumer social app vs fintech, etc).
It's important to first consider what your product and business goals are, and then select metrics that are a useful proxy for that. Here are some of the most common metrics people use to measure product engagement:
Raw behavioral metrics
These are raw numbers from what users are doing with your product and are fairly simple to understand. Examples include...
- Sessions per user (how often they use the product)
- Session length/time in product
- Specific actions completed (clicks, submissions, messages sent, files created, etc.)
- Frequency of use (daily, weekly, monthly active users)
Calculated metrics
These metrics are derived by doing some calculations on the raw behavioral metrics from above. Some examples include...
- Activation rate - % of users who complete key actions that correlate with retention
- Stickiness - DAU/MAU ratio (how many days per month active users actually show up)
- Retention rate - % of users still active after X days/weeks/months
- Churn rate - % of users who stop using the product over a time period
- Engagement score - weighted composite of multiple behaviors
Product Engagement Scores (and why they suck)
There's a popular metric called a Product Engagement Score (PES) that has made its way around the interwebz. It's a metric that mixes multiple engagement signals (e.g., logins, features used, actions completed) into a single percentile score, designed to represent how "engaged" a user is overall.
It sounds nice; a singular score makes for pretty dashboards and short emails. But in practice, it's rarely ever useful. PES is super reductive since you're just mashing together different behaviors with arbitrary weight. You lose a lot of the nuance, context, and meaning that was supposed to go into choosing a product engagement metric to begin with.
For example, a casual user getting perfect value twice a month might score low and be your happiest customer. Also, different segments usually have completely different engagement patterns, but a PES formula treats them all the same.
How to actually measure product engagement
At PostHog, we think the best way for startups to measure product engagement is to index in on a North Star metric. North Star metrics are numbers that are simple to communicate and easy to measure. They're typically one of the raw behavioral metrics from above, like XYZ.
A great North Star metric enables someone to quickly grasp your business's goals because it's an obvious precursor to revenue.
Quantitative tells you what, qualitative tells you why
Especially early on in your startup, don't underestimate the value of qualitatively assessing engagement through session replays and user surveys and just customer support calls/feedback/etc.
What (if any) keywords are we targeting?
product engagement, product engagement score, product engagement metrics