Retention is the key to building a great business.
When your core product experience brings your users back time and time again, you gain some incredible advantages. You spend less on acquisition, learn from your users faster, and start building a community through word of mouth and referrals.
If you can’t retain your users, you’re going to spend more on acquisition while not growing as quickly. You’ll spread distaste for your product, not delight. Soon, you’ll find that there are no more users out there to churn through.
Retention is a simple phenomenon—but you need customer personas to understand why users choose to come back to your product.
Table of contents
- It’s Not Luck—It’s About Understanding Your Users
- Current User Diagnostic
- Creating a Pivot Table
- Filtering & Segmenting Your Users
- Discovering the Drivers of Habit Formation
- Use What You Learn to Shape Your Users’ Behavior
It’s Not Luck—It’s About Understanding Your Users
The difference between a thriving product and a zombie one is that users want to keep using the former. The difference between a thriving product and a world-changing one is that everyone wants to use the latter all of the time—virtually everyone is a power user.
You can use analytics to go from a non-sticky site to a sticky one by delving into how your users behave and what they value.
Introducing Behavioral Personas
The primary route we use to make sense of customer behavior is doing your persona research. You may be familiar with cohorts based on things like date—all those users who signed up in January 2017, for instance. A behavioral persona is similar, but segments users out by patterns in how they use a product.
If you were analyzing a Gmail extension that checks the spelling and grammar of all a user’s emails, then you could imagine a few simple behavioral personas relating to usage right away:
- High-volume emailers: users who process many more emails than the median user
- Low-volume emailers: users who process fewer emails than the median user
You could expand this out to include people who write their emails very quickly, those who spend more time on them, the degree of broadness of the networks people have (whether they email many different people or just a few), and so on—behavioral personas are a very flexible tool.
Current User Diagnostic
Once you have behavioral personas, you want to see how each of them corresponds with user stickiness. The two big goals of studying your current users are:
- Understanding the value users get from your product, and
- Understanding the patterns that lead to retention.
Why current users? Too often marketers want to look at just their new users — or just their churning users. They want to identify whether or not their acquisition is working, or they want to figure out ways to bring back users who are churning or are on the brink of churning.
Figure out what your current users like about your product, however, start building on that, and you’re going to make your site better for everyone.
Example: Mobile Gaming Site with a Social Feature
Let’s say it’s doing okay, retaining an average percentage of its user base. Yet the team is not 100% sure what the core value of the site really is—is it the gaming aspect, or is it the fact that there’s a social layer accessible as well? What keeps users coming back?
You could have three basic personas, each of which is going to exist in some proportion (and have different retention numbers).
- High social only: Users who heavily use social features, but don’t play many games.
- High gameplay only: Users who mainly play games, but don’t use social features.
- High gameplay + high social: Users who both play games and use the social features actively.
The point of studying current users is to build on the core experience that users value and make it even better.
To do that, we’re going to segment these three groups by how likely they are to come back to our site. That will tell us whether it’s the most social, the most gameplaying, or the most social and gameplaying users that tend to come back.
To understand how these three groups differ in terms of their engagement, we’re going to export some raw event data and then analyze it with respect to retention.
Creating a Pivot Table
Hubspot’s Dan Wolchonok illustrates how to do basic analytics with little more than some pivot tables as you can find in Google Sheets or Excel: the first step is getting your raw data.
You could export this from a view you’re interested in exploring in Google Analytics, for example.
For whatever events you choose (clicks, pageviews, etc.) you export the number of times that event was performed on a daily (or weekly, or monthly) basis along with individual user IDs. We might pick events like “play game,” “open chat,” and “invite friend” to capture data on both user gameplay and social activity.
You may have to manually reorganize or transpose the data until you get something that looks like the following (for a single user):
Or the following, for many users:
Now we’re going to create a pivot table. Pivot tables are a powerful tool that let you manipulate large sets of data in order to answer questions. And no, they’re not all that complicated.
The most basic explanation of why we’re interested in the pivot table is that it lets you look at your data in a bunch of different ways that would be hard to do in a regular Excel or Sheets spreadsheet.
To get started, go to Data > Pivot table (if you’re using Google Sheets) and then add your user IDs as your Rows, and your columns as Values.
When you’ve done that, you’ll have a finished pivot table that looks something like this:
Why Use Pivot Tables?
A pivot table lets you perform all kinds of quick calculations on your data:
- count unique values
- calculate variance
- standard deviation
These kinds of calculations can be invaluable for producing reports on your data or exploring trends. Simply sitting in front of a pivot table for a little while, adjusting how your data is being presented can be a great way to clear out those cobwebs and find something interesting to explore further.
Example: Exploring How New Users Become Current Users
We want to know if there’s anything that could be improved about our onboarding, and so we’re looking for users who started visiting our site somewhat regularly and then dropped off.
To see how well you’re doing at onboarding new users, you could investigate those users who return to your site every day for three days after first contact, but not after that.
To do that, you could create one filter in your pivot table that filters out all blank values from Days 1-3. This will reshape our pivot table and show us only those users who came back and performed an action on Day 1, 2, and 3:
You’d then do the reverse for Days 4-5, filtering in the blanks. What you’d get would be a view of all of the users who performed an action on Days 1-3 but not on 4-5 — in other words, a list of IDs for all churning users.
Filtering & Segmenting Your Users
This same technique could be applied to a more sophisticated study of retention — for instance, with brackets of time.
Maybe you run a grocery delivery service and only expect users to come back to your site once every two weeks or so. You could then filter out your columns in two-week increments (with small buffers built in) to:
- Show only those active on Days 1-3, and
- Days 14-16, and
- Days 28-32, and so on
This would give you a more accurate picture of whether or not your site is succeeding in bringing users back.
But both these methods involve taking a static set of users and then narrowing down your view based on patterns in how and when they perform events. You can look at only churning users, for instance, as we demonstrated.
If you wanted to talk to the users who appear to be getting the most value out of your app, you might sort your activity columns based on the number of times an action was performed. Then you’d see only your most active users, and you could follow up with them to figure out what is most important to their high engagement.
You don’t have to look at a static group of users all the time, however. You can also do this kind of analysis on a more specific group of users if you identify particular ones that you want to investigate further.
Using Google Analytics to Segment and Filter
Practically every analytics tool out there will let you segment out the users you want to investigate by demographic or behavioral traits. In Google Analytics, you do this by entering the settings of the view you want to analyze and clicking “Segment”:
From there, you click New Segment and fill out the information that you want to group your users by. In this example, we use Google Analytics’ behavioral segmenting to see only users with 25 or more sessions who have been inactive for 5 or more days:
You could then export this list of users and perform the same kinds of pivot table calculations as we did before. This time, however, you would be seeing behavioral trends specific to these “Churning” users.
Maintaining a Focus on Retention
The real power of this kind of analysis comes in, however, when you combine segmenting users by behavior and looking at their retention.
To go back to our gaming example, we could use Google Analytics’ advanced conditions to create user segments corresponding with:
- High amount of engagement with social features, but NOT gameplay
- High amount of engagement with gameplay features, but NOT social
- High amount of engagement with gameplay AND social
Then, exporting that user data into a pivot table, we could see just how each user segment stacks up when it comes to retention.
In this example (generated in Amplitude), we can see that high gameplay + high social users were clearly the most engaged group. High social and high gameplay were not retained as well, but were still retained better than the overall cohort of current users.
This doesn’t give you proof of anything on its own—but it does suggest that encouraging users to take advantage of both features, social and gameplay, would lead to better overall retention.
How do you figure out what kinds of behaviors to analyze if you don’t already have an idea of what might set your current users apart? You have to discover the drivers of habit formation.
Discovering the Drivers of Habit Formation
Current users have made a habit out of using your product. To understand how they got there, you need to look at your data in terms of behavior—events taken on your site. You need to figure out what kinds of behaviors differentiate your current users from your newest ones.
To qualify an action as a behavioral driver:
- Most users who complete the action(s) should form a habit and become current users
- Most users who do not complete the action(s) churn before becoming a current user
Generally, when someone begins using your product, they will go through several periods before they become a current user. They will start with onboarding, move to value discovery, enter habit formation, and then mature to a current user.
4 Steps to Identifying Behaviors That Correlate with Retention
You determine the location of your habit formation stage by identifying patterns of behavior that tend to correlate with retention. Think of Facebook’s “7 friends in 10 days” benchmark—their observation was that users who added 7 friends within their first 10 days on the platform tended to stick around.
Identifying these patterns of behavior involves some intuition about your product and it involves some analysis of data. You have to compare what users in the habit forming phase do next to what your current users do if you want to truly understand how a user builds a routine interaction with your product.
1. Create a base cohort of users who were retained in the habit formation period. For example, all those who continued to return during Days 4-6.
2. Create a retained cohort of users who were retained in the next period after habit formation (i.e. those who would qualify as current users). For example, all those who continued to return during Days 8-14 to Days 15-28.
3. Create a churned cohort of users who were in the base cohort but not in the retained cohort. All users who returned during Days 4-6 but not Days 8-14 to Days 15-28.
4. Compare retained and churned cohorts to look for behaviors that are present in the retained cohort, but not in the churned cohort. Compare behavior of users who returned during Days 4-6, Days 8-14 and Days 15-28 and those who returned during Days 4-6 but not Days 8-14 to Days 15-28.
You might check for different behaviors by exporting these two cohorts into a spreadsheet and comparing their retention to how often they perform particular events — as in the pivot table described above — though you’ll want to start out by brainstorming the types of events you think may be important drivers.
Talking to Your Customers
Talk to your customers and figure out what your most engaged users seem to value about your product.
Hiten Shah, a SaaS expert and a master of customer development, asks the following survey questions to gauge why a particular product is successful:
- How would you feel if you could no longer use [product]?
- How did you first use [product]?
- What would you likely use as an alternative if [product] were no longer available?
- How could [product] be improved to better meet your needs?
- What is the primary benefit that you have received from [product]?
- What type of person do you think would benefit most from [product]?
The trick to this process is that the first question, “How would you feel if you could no longer use [product]?” triggers a segmentation in responses.
Those who respond they’d be “very disappointed” get put in one bucket, those who say they’d be “somewhat disappointed” get put in one, and those who say they’d be “not disappointed” get put in another. Then, you can start to get an understanding of how differently-invested users of your product differ in terms of behavior.
This is essentially like creating behavioral personas. Your users are not all the same, and you shouldn’t look at them as such. You need to segment them out based on how they behave and whether or not they’re getting value out of your app (whether you define that through a survey question or event data).
Then, armed with some potential drivers, you can start analyzing how your churned users and current users behaved differently.
Use What You Learn to Shape Your Users’ Behavior
As you move into more complex analysis, you should consider key questions, such as:
- What are the key action(s) that you identified as drivers of habit formation? What are some methods you can test to get more new users to cross those thresholds?
- How are your least and most engaged users different?
- Did your behavioral persona analysis reveal any use cases you didn’t expect or didn’t think were very important?
- Are some of your personas more important for your main business objective, like revenue?
These will call for more data comparisons, but also for digging into particular personas. For example, if your main objective is revenue, you could easily compare how much different groups of users are spending.
But if you see people who add three friends on their first visit to a site are better retained, you would need to investigate how users within that persona emerged. Maybe you find that 75% of those adding three friends on the site in their first session were themselves invited to the site by a friend.
Then, driving more users to invite friends could be a more valuable prompt than asking users to find people they know.
Introducing Experimentation and A/B Testing
Reaching a juncture like this is the perfect place to start testing hypothesis to improve user retention. What you have is correlation — not causation — so you need to experiment to find out if your insight into user behavior is real.
You might start by setting up an A/B test where half of your new users who sign up go through an additional onboarding step where they’re encouraged to add three friend, and half do not.
Let’s say you’ve determined that your users go through habit-formation between Day 4-6 and are pretty steady, current users by Day 7.
You could start this test on a Monday, and by the next Monday you would have enough information to see whether the extra onboarding step made a difference. If the segment of users that went through the step are coming back on Day 7 in larger amounts than your control segment, then you have some good evidence for your hypothesis.
You may not yet know that these users are going to stick around for the long-term, but you know that they’re coming back following your habit formation stage — increasing the odds significantly that they will become current users.
This is dependent on having the right sample size of users, so you may have to run the test for longer if your audience is smaller.
Through these sorts of experiments, you can start to uncover the actionable insights that will allow you to drive better retention.
Current users are a window into the best ways to raise your retention rates. They are people who have already realized the value of your product and incorporated it into their routines — formed a habit.
By looking at current users in contrast with other user groups, you can start to reveal the pattern that leads to a user habitually interacting with your product. The basics of building that analysis are:
- Identifying current users
- Finding relevant behavioral personas for your product
- Comparing cohorts to to identify potential retention drivers
- Brainstorming questions for further analysis from your comparisons
- Testing your hypotheses
Building your current user base is important because it’s the foundation of your growth. Although it can be hard to know exactly what drives user behavior, taking the time to tease apart the complexities of different user groups builds your long term value.
Join the conversation
Add your comment
Very good in depth analysis. Content that is presented in a new and original way is more interesting and more likely to receive attention of readers and go viral than plain old text. Eventually, thanks for sharing your healthy experience with us…
Great overview of some retention analysis in an easy-to-understand way that most can apply :)
What about motivation? Did you ever conduct a research and segmentation based on motives?
Comments are closed.