Where NPS Falls Short (and How to Put It to Work) | CXL Institute Revenue optimization services Blog Search Start 7-day trial for $1 Subscriptions Minidegrees Sprints Online courses Free courses Resources

Where NPS Falls Short (and How to Put It to Work)

Where NPS Falls Short (and How to Put It to Work)

Net Promoter Score (NPS) is a valuable customer experience tool, and companies everywhere use it to improve their experience and boost revenue.

But there are many misconceptions and misuses of NPS, and this piece will address those, in addition to showing you how you can actually get value from the tool.

See, when people lean on NPS like it’s a magic number, it doesn’t do much good. It doesn’t help that the original HBR article on NPS called it “the one number you need to grow.”

Unfortunately, the real world is a little more nuanced.

What’s so special about the Net Promoter Score?

Net Promoter Score is a customer loyalty metric developed by (and a trademark of) Fred Reichheld, Bain & Company, and Satmetrix.

It relies on only one question: “How likely is it that you would recommend our company/product/service to a friend or colleague?

Then, you break up responses into three chunks:

  • Promoters (9–10). These are your happiest and most loyal customers who are most likely to refer you to others. Use them for testimonials, affiliates, etc.
  • Passives (7–8). These customers are happy but are unlikely to refer you to friends. They may be swayed to a competitor fairly easily.
  • Detractors (0–6). Detractors are unhappy customers who can be dangerous for your brand, spreading negative messages and reviews. Figure out their problems and fix them.

You calculate your Net Promoter Score by subtracting the percentage of Detractors from the percentage of Promoters.

Instead of distributing bulky customer satisfaction surveys—where even your managers don’t know what to do with the results—you have only one question and one metric to deal with.

Beauty in simplicity

That’s the big benefit of NPS: It’s simple.

Data is worthwhile only if you do something with it—if it spurs action on your part. If you have a bunch of customer success metrics but don’t use them for anything, you’re wasting your time. As Fred Reichheld, creator of NPS, put it:

Most customer satisfaction surveys aren’t very useful. They tend to be long and complicated, yielding low response rates and ambiguous implications that are difficult for operating managers to act on.

When you have an NPS score, you can:

What’s a good NPS score?

Any NPS that is positive is usually perceived as good, and an NPS score of 50+ is considered excellent. The range of NPS is from -100 (all detractors) to +100 (all promoters).

But that’s just aggregate information. It’s not totally useful because it’s not your specific industry. Since NPS is a standardized market research tool, however, it’s possible to compare your score to competitors and within your industry as a whole.

For a simple number, there’s a bit of disagreement over how you gauge whether or not you have a “good” score. But, overall, I like to think of it as a metric to improve internally.

As Adam Ramshaw said on a Quora thread, “a good net promoter score to have is one that is higher than you had last quarter” (kind of like a “good” conversion rate).

Well, what’s wrong with NPS?

Simple and useful as it may be, NPS doesn’t go without criticism—even on our own blog comments:

example of criticism of net promoter score.

Some people clearly aren’t big fans. Prominent market researchers, usability experts, and academics have all voiced concerns about NPS.

The downsides of simplicity

It’s no secret that we like to imagine the world as simpler and more predictable than it actually is. To do that, we construct narratives and frameworks that allow us to make sense of the world.

When you take a short personality test, you get a result that lumps you with all the other ENFPs. When you’re born in September, you’re a Virgo. If you were good at drawing as a kid, you might have been labeled right-brained.

Problem is, the world is more nuanced than that. Lots of research has refuted Myers-Briggs, right vs. left brained, and other lenses you use to view yourself and the world.

When you attach a business metric to “one simple question,” you’re competing with these other simplifications to try to fit a complex network of data into a single number or lens.

Net Promoter Score has been shown to correlate with customer loyalty, retention, and growth—but not always. It does give you an anchor, a number you can move up and down, a sense of progress.

correlation between net promoter score and growth.
Bain research has established a correlation between NPS and growth. (Image source)

You’d think from reading (sometimes sensational) blog posts that NPS is all you need. Throw away analytics and your other customer research tools! This one simple trick will grow your business in your spare time!

It’s a bummer when you talk to survey design experts, or usability experts, or anyone who’s done a lot of market research. They’ll tell you that no one question can tell you everything.

What NPS can’t tell you

According to UserTesting, “NPS can tell you what your customers think of you, but not necessarily why they feel that way about your brand.”

This is true of any quantitative metric, of course. And that’s why NPS tools often ask a follow-up question like, “What can we do to improve?”

The author of that UserTesting piece, Jennifer Winter, goes on to give this example:

Imagine that a company sent out an NPS survey immediately following a customer’s purchase. The purchase experience was good, so the customer gave the company a high score.

But, a few days later, when the customer received the product, it wasn’t what they ordered. The return process was the polar opposite of the checkout process. The customer is now frustrated, upset, and has vowed never to make another purchase. If another NPS survey were sent out at this time, chances are this customer would be a vocal detractor.

NPS captures just one point in time with a customer, and the customer’s response will depend heavily on their most recent experience. Focusing solely on NPS as a measure of overall CX is a dangerous habit that could eventually turn loyal promoters into detractors.

There are also NPS-specific limitations in terms of branding and user experience measurement. Perhaps the best example I’ve heard is from a talk by Jared Spool:

I was so disappointed when the people at Medium sent me this: “How likely are you to recommend writing on Medium to a friend or colleague?” It’s not even a 10-point scale. It’s an 11-point scale, because 10 was not big enough.

This is called a Net Promoter Score, and Net Promoter Scores, if you look at the industry averages that everybody wants to compare themselves to, the low end is typically in the mid-60s, and the high end is typically in the mid-80s.

You need a 10-point scale because, if you had a 3-point scale, you could never see a difference. Anytime you’re enlarging the scale to see higher-resolution data it’s probably a flag that the data means nothing. Here’s the deal. Would a Net Promoter Score for a company say, like United, catch this problem?

Alton Brown bought a $50 guest pass to the United Club in LA and had to sit on the floor. I wonder what his Net Promoter Score for that purchase would be? It probably wouldn’t tell anybody at United what the problem is.

But that’s a negative. What about the positive side?

What’s actually working well? Customers of Harley-Davidson are fond of Harley-Davidson, so fond that they actually tattoo the company’s logo on their body. This is branding in the most primal of definitions.

NPS isn’t useless, but it can be dangerous because you have one data point. And you think you know your audience because you have a data point, but don’t realize how nuanced their interactions with your company actually are.

Take, for example, Caroline Jarrett’s (author of Forms That Work) discussion on how certain products just aren’t as “recommendable” as others:

Just from the point of view of using a Net Promoter Score as a question in a survey, we have to ask whether that question means as much to the people answering it as it might to the business. There are some things where “I’ll recommend this to a friend” is a really important thing that people would actually do. But there are other things where you’d never recommend it to a friend because you don’t do recommending, and you certainly don’t do recommending of those type of things.

So you might actually be very enthusiastic about the product, but you just might not ever feel the urge to recommend hemorrhoid cream to your pals. You know? That’s not then giving a true measure of the value of that product. I have my skepticism about Net Promoter Score.

Other common criticisms of NPS hinge on its accuracy: Does it actually model loyalty? Does it actually predict growth?

Some claims people make:

NPS and predictive validity

Though the original research that led to the development of NPS showed strong evidence that NPS correlated with growth, studies since haven’t done much to support it.

In fact, studies have shown that other indicators predict growth and represent customer loyalty more accurately. NPS has much room for improvement in terms of validity.

For example, one study claimed that:

Recommend intention alone will not suffice as a single predictor of customers’ future loyalty behaviors. Use of multiple indicators instead of a single predictor model performs significantly better in predicting customer recommendations and retention.

Another similarly stated, “given the present state of evidence, it cannot be recommended to use the NPI as a predictor of growth nor financial performance.”

While it makes intuitive sense that NPS correlates with growth (people are more pleased with a company, they tell their friends, more people buy stuff, etc.), sometimes it’s not so clear cut.

In fairness, even Reichheld said that NPS doesn’t always predict growth:

The “would recommend” question wasn’t the best predictor of growth in every case. In a few situations, it was simply irrelevant…

…Not surprisingly, “would recommend” also didn’t predict relative growth in industries dominated by monopolies and near monopolies, where consumers have little choice. For example, in the local telephone and cable TV businesses, population growth and economic expansion in the region determine growth rates, not how well customers are treated by their suppliers…

…And in certain cases, we found small niche companies that were growing faster than their net-promoter percentages would imply. But for most companies in most industries, getting customers enthusiastic enough to recommend a company appears to be crucial to growth.

But it’s not just in fringe cases and industries. NPS has been shown to have low”predictive validity,” meaning the level of value a scale has in predicting future business performance. A study showed that NPS actually had the lowest predictive validity of four scales tested.

That study reinforces Spool’s belief that “Net Promoter Score is an ineffective instrument for measuring how your customers feel about you.”

Therefore, it’s pertinent to figure out correlative metrics that are specific to your business. Maybe it’s NPS, maybe it’s something else. But simply relying on NPS as a crystal ball is no way to forecast future growth.

NPS and cultural variance

CustomerGauge wrote about this, dubbing it the “Dutch Effect.” (The Dutch won’t give you a nine or 10, I guess.) They suggest, however, that this has nothing to do with cultural perceptions of score ratings. Rather, the Dutch are used to getting poor service, so they rate poorly.

I’m not sure that’s the case. Even Rob Markey, Bain & Company Partner and co-author of The Ultimate Question 2.0, said that cultural factors are at play:

Some cultures, for example, use response scales differently. In some Asian countries, for example, few customers are rarely willing to use the top end of any scale. In some other countries, customers use the extremes of any response scale, but don’t use the middle very often. Comparing absolute scores in this instance is nearly impossible.

How to get actual value from NPS

A Net Promoter Score and a pile of comments isn’t worth a dime if you don’t plan to put them to work.

Jessica Pfeifer, Co-founder of Wootric (source)

Net Promoter Score, for all the criticism it receives, is still very useful. As a dormant number or an industry comparison, it doesn’t do much for you. But as part of a fluid and constant operational management process, NPS brings great value.

Jeff Sauro from MeasuringU puts it really well:

The Net Promoter Scoring system certainly has its problems and my friends at Bain and Satmetrix have probably oversold it. Yet, despite its problems…the NPS works fine as a measure if you understand the shortcomings and make some adjustments.

It’s never a good idea to put all your measurement eggs in one basket. Consider multiple measures of both satisfaction and loyalty. You should also look to understand how well the NPS correlates (or even predicts) revenue and growth in your organization and understand how other measures may do a better job.

If you’re looking to implement NPS, here are four best practices for getting the most value from it:

  1. Ask follow-up questions.
  2. Combine it with user research.
  3. Find and fix issues.
  4. Market to promoters.

1. Ask follow-up questions.

Seeing an improvement in your aggregate NPS is nice, but the real value comes from the feedback in follow-up questions. At least that’s how Sachin Rekhi put it, writing on Andrew Chen’s blog:

The most actionable part of the NPS survey is the categorization of the open-ended verbatim comments from promoters & detractors. Each survey we would analyze the promoter comments and categorize each comment into primary promoter benefit categories as well as similarly categorize each detractor comment into primary detractor issue categories.

Wootric also emphasizes automating follow-up questions to get qualitative feedback along with your quantitative NPS. They suggest the following three follow ups:

  • Promoters. What’s your favorite part about our product/service?
  • Passives. What would make you love us?
  • Detractors. What could we do to improve your experience?

2. Combine it with user research.

As mentioned before, NPS isn’t enough data to base decisions on. Adding a follow-up question opens the door to valuable qualitative feedback and voice of customer research.

But you also can’t neglect user research, including usability testing and other common conversion research techniques.

This will help you find specific problems that correlate with low NPS scores. As UserTesting said:

You may discover that although your NPS is high, nearly every customer was having the same issue with your navigation, or that the copy on your pricing page doesn’t clearly explain what’s included in the price.

Some techniques that can help pinpoint these usability issues:

3. Find and fix issues.

NPS, though commonly used as a customer success tool, can also be used by other teams, including your optimization team. Combining the overall score with a follow-up question, as well as the rest of your conversion research, will help you build and prioritize test ideas.

NPS, depending on the time you ask the question, doesn’t necessarily correlate with on-site user experience. But NPS could and should include more of the whole experience—customer service included—so that optimization teams can work on issues that customers voice on the survey.

There’s a huge opportunity to turn detractors into promoters. As Reichheld said, “Every detractor represents a missed opportunity to add a promoter to the customer population, one more unpaid salesperson to market your product or service and generate growth.”

Omniconvert also wrote about NPS and how you should deal with detractors:

The best way to go with detractors is to investigate why they had an unpleased experience and why they will not recommend you. Knowing the answer, you can solve and improve that certain aspect and, in this way, transforming a detractor into a passive customer or even a promoter. Make them an offer they can’t refuse!

4. Market to promoters.

Segmenting your audience into detractors, passives, and promoters can help with your marketing efforts as well. One of the most popular ways to do that is to turn promoters into advocates. As Wootric put it, “When a customer rates your company at nine or 10, it’s the perfect time to ask them to refer friends.”

Lincoln Murphy talks about using NPS to evangelize your best customers. Just because they say they’d recommend you doesn’t mean they jump up and do so. You’ve gotta nudge them:

Whether that’s getting them to write a review for you (or you write it for them based on your knowledge of the success they’ve had with your product and get them to sign-off on it), or to talk to a prospect, get their logo on your site, etc.

Having a very clear workflow for acting on Promoter results is huge. Generally it’s some type of escalated advocacy ask…first it’s a quote, then it’s a case study, then…etc.

Whatever works in your world with your customers.

Sujan Patel employs a similar strategy with promoters:

I use NPS…to turn my promoters into advocates. After all these customers are ready to promote you they just need to be told how or a small nudge. I offer these customers swag (free t-shirts, mugs, etc) and ask for help they can do quickly such as share on Facebook or Twitter, write an app store review, email a friend.

Even if it’s not explicitly asking for a recommendation or giving away free stuff to promoters, there are ways to analyze your NPS data and find correlative actions that may predict success.

As Sachin Rekhi put it:

We correlated specific behavior within the product to NPS results (logins, searches, profile views, and more) and found a strong correlation between certain product actions and a higher NPS. This can help deduce what your product’s “magic moment” is when your users are truly activated and likely to derive delight from your product. Then you can focus on product optimizations to get more of your customer base to this point.

The best way to get to these correlations is simply to look at every major action in your product and see if there are any clear correlations with NPS scores. It’s easy to just graph and see if this is the case.

Lincoln Murphy also recommends using NPS as a growth-hacking tool, basically triggering Consistency Bias to reinforce positive beliefs:

Survey the customer immediately after they did something that either resulted in them achieving value or in them getting closer to achieving that value from your product.

When you trigger the “how likely are you to refer somebody…” while they’re in a positive frame of mind, not only will you reinforce their positive feelings toward you, but when they identify as a “promoter,” they’ll also be reinforcing that self-identification.

Conclusion

While many blog posts and consulting firms that tell you NPS is the be-all-end-all metric, know this: No single survey question can predict your company’s success.

Studies also suggest that NPS isn’t as predictive of growth, retention, or virality than initial research claimed.

Either way, NPS isn’t all bad. If anything, it triggers an organization-wide attitude toward improving the customer experience. There’s nothing wrong with that.

And, if used in conjunction with user research, analytics, and A/B testing, NPS can be a solid addition to the conversion research arsenal.

Feature image source

Related Posts

Join the Conversation Add Your Comment

  1. As a brand builder concerned with CRO, I pride myself on being current with new SEO, SoMe and other marketing trends. So when I see an email from my peep PEEP (as in Peep Laja) with some acronym I don’t know like NPS, it gets my attention. Yes, pitching to your supporters usually works better than serenading the critics. :) Glad to know what Net Supporter Score is! Nice article, Alex.

    1. Alex Birkett

      Thanks, Ann – glad you enjoyed the article :)

  2. Really nice article Alex. I think you have produced a balanced view of NPS, highlighting how it can be useful and the pitfalls of relying totally on a single number/metric. We realised a long time ago that NPS had limited application and devised our own metric called Strength of Relationship Index (StoRI) and incorporated this and some other smart stuff into our SaaS platform (www.valuiza.com).

    1. Alex Birkett

      Yep, anytime someone tells you there’s “one easy trick” or some variation of that, they’re probably selling you something (and Bain’s doing a good job with NPS). Interesting that you developed your own metrics. Working out well for you?

    2. It’s early days working with our own metrics but we are getting plenty of interest from our users because they recognise the limitations of using NPS by itself. I should add that we have also developed IP that identifies the quantitative changes in outcomes such as retention and advocacy based on changes to underlying drivers and we can link this to predictions of customer lifetime value to demonstrate bottom line impact.

  3. I’d like to pick up on the problem that Jennifer Winter described and turn it (jiu-jitsu style) into an advantage:
    <“NPS captures just one point in time with a customer, and the customer’s response will depend heavily on their most recent experience. “

    As I see it , the problem with many satisfaction/brand relation metrics and methods is that they are too loosely connected to any customer experience. So you can sit day- in, day-out and stare at your customer satisfaction metrics or brand awareness metrics and still you won’t have the answer to the question – “What should I do in order to improve these metrics?”

    So where NPS is limited in terms of reporting to the execs, it can be gold for us optimizers!

    If your are able to connect the NPS survey to an actual experiment, such as an AB-test or a personalization tactic, you will get additional data out of your experiment. When people exit the experiment you can measure how much the experience affected their NPS score. And this data will not only tell you how many clicked or bought, but additional qualitative data on WHY they did so.

    And the answer to the question WHY, is the most valuable answer you can get from any customer.

  4. Great overview of the net promoter score, Alex. I agree in that the NPS should not be viewed as the sole measurement and generally I think surveys with both qualitative and quantitative measures are necessary, especially in marketing. What I found very valuable was the section on converting promoters and splitting marketing based on which score was given. It shows how NPS can be useful in strategic planning and does not need to be removed completely, just improved and updated.

    1. Alex Birkett

      The marketing use case for NPS is, I think, the most useful as well. As an industry benchmark, I think it’s overblown. Though there are awesome ways to incorporate the score into your larger growth optimization strategy (see John’s comment above)

Comments are closed.

Current article:

Where NPS Falls Short (and How to Put It to Work)

Apply for the Turning Data to Insights sprint

Join this 6-week live, interactive training program to learn how to extract the right insights from your data, so you can take data-informed action.

Apply now

We’re running the CXL Sprint program in three other disciplines:

  • CRO program management
  • Growth experiments
  • Differentiation

Sign up here to stay updated with upcoming program dates and enrollment info.

Categories