Learning More About That Other Half: The Case for Cohort Analysis and Multi-Touch Attribution Analysis (Part 1 of 2)

Note: This article first appeared as a guest post on the popular KISSmetrics blog. Thanks to Hiten Shah and Sean Work at KISSmetrics for publishing it. I'm republishing the post here as a series of two shorter posts, with a few small edits.

Anyone who has ever worked in marketing or advertising has heard the quote, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” It is from John Wanamaker and dates back to the 19th century.

Fortunately, the industry has come a long way since then, and especially in the last 10 to 20 years, new technologies have made advertising more measurable than ever. However, there’s still a considerable gap between what people could measure and what they actually are measuring, and that leads to significant under-optimization of advertising and marketing dollars.

In B2B SaaS, which we at Point Nine Capital focus a lot of our efforts on, there are two techniques that I feel are particularly important but not used widely enough – cohort analysis and multi-touch attribution analysis. In this series of posts, I’ll try to provide a brief introduction to both methodologies and explain why I think they are so important.

A Quick Primer about Cohort Analysis

If you're a reader of this blog or know me a bit, you know that I'm a huge fan of cohort analysis and have written about the topic before. If you’re new to the topic, a cohort analysis can be broadly defined as a dissection of the activities of a group of people (such as users or customers), who share a common characteristic, over time. In SaaS, the most frequently used common characteristic for grouping customers is “join date”; that is, people who signed up or became paying customers in the same period of time (such as a month).

Let’s look at an example, and it will become much clearer:

In this cohort analysis, each row represents all signups that converted to become paying customers in a given month. Each column represents a month in your customer’s life. The cells show the percentage of retained customers of the respective cohort in the respective “lifetime month.”

So What?

Why is it so important to do a cohort analysis when looking at usage metrics or retention and churn? The answer is that if you look at only the overall numbers, such as your overall churn in a calendar month, the number will be a blend of the churn rate of older and newer customers, which can lead to erroneous conclusions.

For example, let’s consider a SaaS business with very high churn in the first few lifetime months and much lower churn from older customers – not unusual in SaaS. If the company starts to grow faster, the blended churn rate will go up, simply because the percentage of newer customers out of all customers will grow. So, if they look at only the blended churn rate, they might start to panic. They would have to do a cohort analysis to see what’s really going on.

What else can you see in a cohort analysis? Whatever the key metrics are in your particular business, a cohort analysis lets you see how those metrics develop over the customer lifetime as well as over what might be called product lifetime:

If you read the chart above (which I've borrowed from my colleague Nicolas) horizontally, you can see how your retention develops over the customer lifetime, presumably something that you can link to the quality of your product, operations, and customer support. Reading it vertically shows you the retention at a given lifetime month for different customer cohorts. This might be called product lifetime, an, especially if you look at early lifetime months, it can be linked to the quality of your onboarding experience and the performance of your customer success team.

The Holy Grail of SaaS!

Maybe most importantly, a cohort analysis is the best way to estimate CLT (customer lifetime) and CLTV (customer lifetime value), which informs your decision on how much you can spend to acquire a new customer. As mentioned above, churn usually isn’t distributed linearly over the customer lifetime, so calculating it based on the blended churn rate of the last month doesn’t give you the best estimate. A better way is shown in the second tab of this spreadsheet, where I calculated/estimated the CLT of different cohorts.

A cohort analysis is even more essential when it comes to CLTV. Looking at how revenues of customer cohorts develop over time lets you see the impact of churn, downgrades/contractions, and upgrades/expansions:

This chart shows a cohort analysis of MRR (monthly recurring revenue) of a fictional SaaS business. As you can see in the green cells, it’s a happy fictional SaaS business as it has recently started to enjoy negative churn, which many regard as the holy grail in SaaS.

Still not convinced that you need cohort analyses to understand your SaaS business? :-) Let me know in the comments.