We humans live in the present. We tend to value 10 EUR today more than 10 EUR in five years. This is the same in business. Usually, we chase short- to mid-term goals.
The catch with Anything-as-a-Service models is that the impact of many decisions will only materialize in a few months or years. So, it needs time before a change can be classified as success or failure.
Take, for example, a subscription on shoes. Right now, customers stay with the service for an average of 5 months. We want to increase the average contract duration to 12 months. We assume that changing the minimum contract from 3 months to 6 months will get us there. The price remains the same, so we want to scare of customers who intentionally want to stay shorter than one year. When do we know if we achieved that goal? Right, only after about twelve months. We might know it a bit before (if everyone cancelled already) or a bit later (if many subscribers are still active), but it needs time.
During that time, we certainly implemented other changes. We ran a couple of campaigns, did some partnerships, changed prices, optimized the portfolio, and a new competitor entered the market – you name it. How do we know twelve months after the increase of the minimum contract duration from 3 months to 6 months if our hypothesis turned out to be true?
The answer is cohort analysis. Cohort analysis is a method to look at data of different chunks. A cohort can be customers who signed up at a specific week. This cohort is compared to users who signed up the weeks before and after. A cohort can also be customers who came in via a special promotion (e.g. identified via a promo code) versus all other customers who signed up during that time. In our shoe example, we compare the customers who signed up in the weeks after we increased the minimum contract duration to 6 months with those who signed up the weeks before.
There can be many criteria that constitute a cohort:
- date: customers signing up after a specific date
- channel: customers acquired via a specific channel, e.g. affiliate campaign
- geography: customers who signed up in one country or city, e.g. where a distinct offer was made
- payment method: customers who chose credit card as the payment method
- basket size: customers with a basket size above a certain value
- product type: customers who signed up for a specific product cate
- sociodemographic aspects: customers of a certain age, shipping address, sex, interest or so
- etc.
If you start doing that, you will quickly combine dimension. The starting dimension is always date: a certain change in the proposition was made at a certain time, therefore you look at a certain customer segment before and after a specific date. Looking at the customer satisfaction of customers acquired via a distinct campaign in one city signing up to a special service can be another narrow cohort. This will reveal more valuable insights than just looking at the customer satisfaction of all customers who signed up during the campaign period.
The cohort analysis will give you insights in the behavior of customers who are at the same point in the customer journey and share at least one more characteristic. Instead of looking at all customers – no matter their stage in the customer journey – the cohort analysis creates more transparency about the impact of changes of your acquisition, conversion, retention, and other improvement measures.
Soon you will run into at least two challenges:
First, my business is not old enough to run this analysis. If you just started and your service requires customer lifetime to be many months or several years, you cannot answer these questions yet. This is a problem by design.
However, there is one way that can reduce that challenge: ask the customers. This works for data points customers can and want to reveal. Let‘s take the example of shoes again. During your onboarding, you can ask new customers, „How long do you intend to stay?“ Customers will provide an estimate. This won‘t be accurate, of course. It might be biased as well. However, if you started asking this question from the very beginning – before implementing the change you want to analyze – you can monitor trends. After you change the contract duration, you‘d expect to see changes in customer replies right away. If you don‘t see them, odds are high that you won‘t see the desired outcome in twelve months from now. This workaround does not work for all changes. Imagine you change your payment process to reduce outstanding invoices. I would not expect new customers to tell me accurately how their payment behavior will be.
Second, I have gaps in my data to answer these questions. Imagine you feel that some marketing campaigns bring in more valuable customers than others. A valuable customer can be determined by the customer’s lifetime value, the referral rate, or a customer satisfaction score. It‘s fair to assume that some campaigns might tap into customer pools that love your product whereas others just want to try but leave relatively soon. The immediate conversions might be identical, but the long-term value is different. Obviously, you are interested in the campaigns with a higher long-term value. You want to run such an analysis, but you realize that you are not able to attribute customers to specific campaigns. There is just one solution to that problem: start collecting this data, enable the relation to other customer data, and make it accessible for your analysis.
If your business is relatively young, you might want to take data from adjacent services or make assumptions. However, adding data from other sources to your cohort analysis should be considered wisely. In many cases, cohort analysis is on such a detailed level that the similarity can be put into question. And making assumptions is practically just a mirror of your own expectation that drove the initial change. So, this will only lead to self-fulfilling prophecies in your cohort analysis. You need assumptions, for sure, but not to validate your changes.
Similarly, to any other analysis, you must ensure that your cohort analysis complies with GDPR or any other privacy framework relevant to your business. You need to collect data and you process data with a certain intent. Make sure from the very beginning, that you get the consent of your users to run such an analysis.
If you do so, you can learn if 10 EUR today will be worth 20 EUR tomorrow.