Cohort Analysis: A (practical) Q&A [Guest Post]

My colleague Nicolas wrote a great guide with tips and tricks on how to do cohort analyses which I'd like to share with the readers of this blog. Thanks, Nicolas, for allowing me to guest publish it here. Without further ado, here it is!

- - - - - - - - - -

At Point Nine we believe that the only way to get a real sense of user retention and customer lifetime is doing a proper cohort analysis. Much has been said and written about them and Christoph has a published a great template and guide on the topic if the concept is new to you.

With this Q&A I want to focus on some of the more practical questions that might arise when you are actually implementing a cohort analysis for your startup. After close to two years of working with SaaS companies and doing numerous of these analysis I have learned that in most cases there is no perfect step-by-step procedure. But although you will always have to do some customisation for a cohort analysis to perfectly fit your business, there are a handful of questions and pitfalls that I have seen over again and again and want to share so that you can avoid them.

Now let's get into it!

Q: Which users should I include in the base number of the cohort?

There are two parts to the answer as it depends on what you want to measure. If you want to find out your overall user retention and have a free plan, then you should include all signups of a specific month.

However if you are trying to calculate your customer lifetime value, you should only look at the number of paid conversions. I only count an account as a paid one when the user has or will be charged for a period. So if you offer a 30-day free trial for example, wait to see if the user converts into a paying plan before you include him in the cohort. This way the numbers won't be biased with users that actually never paid for your service.

If possible without too much effort, you should also try to eliminate all 'buddy plans' that you have given to friends, your team or investors. If they are not paying, they are not representative for the real cohorts.

Q: How do I treat churn within the first / base month?

There are different approaches here, but in my view taking churn within the first month into account is the most accurate representation of reality. That means that in your first month the retention could be less than 100%, if people cancel their paid subscription within that month. It would look something like this:

I do this because I don't want the analysis to exaggerate churn in the second month and understate it in the first / base month. After all the reasons for churning in the first 1-4 weeks could be very different than after 5-8 weeks.

Q: Should I treat team and individual accounts differently?

If you are at a very early stage or sell mostly (90%+) individual plans it is probably sufficient to mix them all in the same analysis. But when team plans make up a significant part of your paid accounts, or your product has a very different user experience when a whole team uses it, you should probably look at both type of accounts separately.

Findings could include that team accounts are a lot more active, churn less and see a lower drop-off in the first month than individual plans. Or not. :)

Q: What about annual vs. monthly plans?

Again, if you are focusing on how active your users are over their lifetime it is OK to mix both plans. If you just want to see how many of the people that signed up still come back after X months, no need to split hairs.

If you are however focused on churn, you should only look at paid accounts that could have churned in that month. This is one of the 9 Worst Practices in SaaS Metrics and means that you should exclude all annual plans that are not expiring in the respective month. Including these in the denominator would otherwise skew churn numbers.

Q: Now that I have it, what can I take away from it?

The two most obvious take-aways are depicted in this (KISSmetrics) retention grid. Note that this is a most likely an analysis for a mobile app and the numbers for your SaaS solution should be significantly higher:

(click for larger version)

Moving horizontally you can see how the retention of a cohort decreases over the users lifetime. Interesting here is where the highest drop-offs occur and whether the numbers stabilise after a few months.

Vertically, you can (ideally) see how the retention of your cohorts change over the product lifetime. Assuming you are not twiddling your thumbs while catching up with House of Cards or sipping Mai Tai’s at the beach once your product launches, you should see an improvement in user retention with younger cohorts as the product improves. If this is not the case, you should consider whether the hypotheses or features you are working on are the right focus.

Most importantly though, this data will be the basis to give you a sense for your customer lifetime value (CLTV). If you take the weighed retention data for the 6th or ideally 12th month and extrapolate it, you will get an approximation for the average lifetime of your customers. Multiplying this with the average revenue per account (ARPA) or respective plan that you are looking at (e.g individual / team) it will give you your CLTV. This number is really the quint essence of the cohort analysis, as it gives you an idea about how profitable your business model is (=how much more money are you making with than what you are paying to acquire him). Subsequently it will also tell you the highest price you can spend on customer acquisition to grow profitably. It is important to note here that although super valuable, especially in the early stages of a startup this number will always be an estimation and most likely not 100% accurate. So keep in mind to continually track and fine-tune your CLTV calculations.

And one last thing: If you have accounted only for paid subscriptions as defined at the first question above, then the base rates of each month will also give you the most accurate number for paid customer growth and subsequently MRR growth. Two charts you will want to have at hand when talking to investors.

Q: Is that it?

For this post, yup! If you want to learn more about cohort analysis or SaaS Metrics, I would strongly suggest to check out Christoph’s and David Skok’s blog. And in case you have any questions on the above or something is unclear, feel free to ask away in the comments or send me a mail and I will do my best to answer you (or forward the hard questions to Christoph). ;)

- - - - - - - - - -

Like this post? Make sure you add Nicolas' blog to your reading list.