I can’t take seeing yet another article extolling the virtues/lambasting the shortcomings of NPS. So I want to lay it all on the table – the pros and the cons – and try to offer a balanced assessment that neither canonizes nor demonizes Fred Reichheld, NPS, or its adherents or critics. Here are my ground rules.
-
- Objective: move beyond the near-religious arguments between the apostles and cynics and present facts, experiences and conclusions regarding NPS
- Approach: describe, analyze, and discuss the relative merits and demerits of NPS
- Planned outcome: a balanced presentation to provide researchers and marketers with meaningful, thoughtful context for evaluating NPS and deciding whether and when to use or not use NPS
- Prayer: pre-empt some of the seemingly never-ending back-and-forth on this issue and glib arguments pro and con
Audacious? You bet, but not any more audacious than staking claim to “the ultimate question.” No other issue in the domain of Voice of the Customer (VoC) – including both customer loyalty and customer experience – is as pervasive or divisive, so it is worth the effort. Let’s jump in with the pros and cons . . .
Pro: Simplicity. Gone are the days of black-box, convoluted metrics. Virtually all CX metrics rely on basic computations and present scores as a single number. In this sense, most metrics are “simple.” NPS is simpler in that it is single variable. Any multivariate measure is more complex than a univariate measure. Promoter sounds much better than “top 2-box” and there never is even an attempt to explain the bottom 6-box classification of Detractors. What can be simpler than answering the only question a company needs to know and classifying people as for, against and in-between?
Pro: C-Suite Credibility & Support. In a stroke of strategic marketing genius, Reichheld and Bain sold NPS at the C-suite and board level. I have never had a client say my CEO, senior leadership or the board likes a particular CX metric – except for NPS, which regularly is adopted specifically because of an edict from above. This acceptance at the C-suite and board level has prompted increased budgets for all types of VoC work. It is easier to get funding for projects embraced by senior execs and linking compensation to NPS (as some companies do) has further solidified resources for VoC projects. While NPS may have stolen the thunder from other metrics and may or may not be better for research and marketing than other approaches, NPS has been a lightning rod for galvanizing the involvement and funding support of executive leadership.
Pro: Popularity. Except for “overall satisfaction,” there is no CX metric used and cited more often than NPS. There are a number of research firms with no IP of their own that focus exclusively NPS, and there is a mini-industry centered on NPS, with its own training, certification and conferences. As a by-product of this popularity, there always is NPS data against which firms can quasi-benchmark their performance. While benchmarks often are misleading, misused, and over-rated, they also are inherently appealing.
Con: Predicting Customer Behavior and Business Outcomes. The validity of any metric depends on the extent to which it explains customer behavior and business results, the outcomes that matter. While every company would prefer to have happy customers (however defined) than unhappy customers, the rationale for investing in CX is that it is good for business: a company captures a greater share of the lifetime value from more loyal customers than from less loyal customers. The “best” CX metric, as such, is that which can be empirically demonstrated to be the best predictor of customer behaviors and associated business outcomes.
Here the evidence for NPS is mixed. Having conducted and read any number of analyses comparing NPS with other CX metrics, in no study I have ever read, seen, or conducted did NPS outperform all other metrics. In no instance did NPS beat the multivariate or any of the other leading loyalty metrics; in some instances NPS proved a better predictor than simple satisfaction and other single variable measures, but not always. If better research and marketing means more valid measures that are more predictive of more desirable customer behaviors, then NPS gets at best a fair to middling grade on this criterion.
This isn’t surprising, as the underlying question behind NPS (likelihood to recommend) is embedded in one form or another in most of the multivariate loyalty metrics: even if NPS is the single most important question combining willingness to recommend with any other relevant variables almost inevitably will have more explanatory power than recommend by itself.
Talk about setting the bar high. I guess “the ultimate question” sounds more compelling than offering “another darn good question,” but NPS fails to live up to the ultimate hype. NPS holds up well against overall satisfaction and other univariate metrics, such as Customer Effort, but falls far short of being distinctively more powerful, let alone attaining “ultimate” status.
Neutral: Moving the Needle. Part of the allure of NPS for senior executives is the promise that all you need to do is ask one question, which sounds like a great way to streamline research (and associated budgets and staff). This claim is a bit of a bait-and-switch. Even if NPS is the ultimate measurement of the customer experience, any metric by itself is anemic, as it begs the question of why the score is what it is and how it can be improved. Knowing the temperature of the patient may be important, but the objective is to improve the patient’s health.
For this reason, every serious NPS program typically uses some form of key driver analysis to identify the performance criteria “driving” the metric. The approach to the driver analysis may be more important than the actual loyalty metric, as the driver analysis — which identifies, quantifies, and prioritizes the areas on which companies need to improve performance to boost their overall loyalty scores – is the actionable component that brings the metric to life. The “ultimate” metric without an understanding of the key drivers is far less actionable than a less-perfect metric linked to an insightful driver analysis.
Mixed: What Defines “Best?” A compass points to “magnetic north,” while the North Star illuminates “celestial north.” This distinction may be immensely important for some technical applications, but both will lead “north” for the layman, both are “good enough” to provide general direction. If the “true north” in CX measurement is determined by the metric that best predicts customer behaviors and business outcomes, then NPS misses the mark. But this doesn’t mean that NPS isn’t a useful metric. NPS points in the right direction, and it is positively correlated with customer profitability and other business outcomes. Improving NPS scores will never be bad for a company.
All thing being equal, it’s difficult to argue for “good enough” when better metrics are available. But all things aren’t equal. If the CEO prefers and the CFO will fund NPS work, if there is deep organizational commitment to and a history with NPS, judgment might dictate that continuing to use NPS is more prudent than starting new trend lines, having to educate leadership and employees, and spending resources fighting over the issue. In such circumstances, an ingrained good metric may make more sense than a better new metric.
Mixed: Industry and Business Model Matters. The appropriateness of NPS depends on the sources of customer value and varies by industry and constituency. Is value generated more by retention (ongoing purchases), cross-selling (buying other products or product lines) and upselling (buying more expensive products and services) or by recommendations to others? NPS is far less appropriate for a supermarket – characterized by frequent purchases and a wide variety of products and product lines – than for mortgage banks and real estate companies, where the lifetime value of a customer has more to do with potential referrals and recommendations than their next house purchase in eight or ten years. In industries such as these, NPS might be a better indicator of loyalty than for businesses where lifetime value is more about retention and cross-selling, such as retail banking, mobile phones or cable/Internet services.
While the value of a recommendation almost always is exceptionally difficult to monetize, recommendations are a component of the lifetime value of customers in virtually every industry. In some instances – such as businesses based on what-ever-of-the-month clubs, sales “parties,” membership models or affinity groups – recommendations are especially important to fueling growth.
There are, of course, other metrics that capture recommendations – such as Advocacy and measures of actual recommendations rather than likelihood to recommend. The more a company relies on word-of-mouth, the more important it is to have an NPS-like metric or NPS-like component in a larger CX metric.
Transactional vs. Relationship Measures. Loyalty is a relationship concept, and NPS – by design – is a relationship measure. All too often, however, NPS is used in conjunction with surveys of various types of customer interactions where the metric simply doesn’t fit especially well. While there may be some instances where NPS might work as a measure of a specific interaction, in most instances the concept of “transactional NPS” is an effort to measure the whole based on an individual part (or a component of a part). In research-speak, this is a “level of analysis” problem and involves relying on an inductive approach that is far better suited to qualitative inquiry than quantitative research.
Every experience, every interaction is part of the overall customer relationship. The whole (or the relationship), however, is more than the simple sum of the parts (or experiences and interactions). Asking a customer if they are likely to recommend a brand because of a particular inquiry to a call center, a specific visit to a branch or a store or a website, or any other fleeting interaction usually is a mistake. The proof of this erroneous approach is apparent from the key driver models and validations of transactional NPS projects: the resulting key driver models of transactional NPS almost always are far weaker than those derived from relationship models, and NPS based on transactional surveys almost always are less predictive of customer behavior than scores based on relationships.
Weakness: The “Net” in Net Promoter. Nets are easy to calculate and appealing in simplicity, but they are statistically troubling. There are any number of combinations of Promoters, Detractors and Passives that yield the same NPS. It’s hard to treat the results as equal when comparing an NPS of 20 based on 40% Promoters, 20% Detractors and 40% Passives with an NPS of 20 based on 60% Promoters and 40% Detractors.
Because of the net calculation approach, moreover, technically the margin of error (MOE) can’t be calculated in the same manner as simple percentages, and univariate measures are inherently more likely to be unstable and have larger MOEs than multivariate metrics. Most CX modeling, moreover, is a customer-level exercise. But NPS is an aggregate measure. An individual respondent doesn’t have an NPS score. Drivers or stated reasons can be used to explain what prompts someone to be a Promoter or a Detractor, but the modeling doesn’t explain NPS per se. While this may seem like the splitting of a technical hair, it is important for researchers and marketers to understand the nuance. CX modeling is conducted at the customer level, while NPS is not a customer-level metric; key drivers explain the motivations behind individual loyalty, not overall NPS.
The Net-Net
So what is a serious researcher and marketer to do?
Weigh the pros and cons of NPS and other approaches with a critical eye. Carefully consider your company and business model. Keep in mind the ultimate business objective: boosting the customer experience and strengthening customer relationships as a path to improving customer profitability. I always like to test multiple metrics to determine which measure best explains or predicts customer behavior and business outcomes.
If your focus is transactional, NPS almost certainly is not appropriate. If your target is the customer relationship and loyalty, NPS always will be a viable approach for consideration – it unquestionably will be “good enough” and may well be the most practical solution. Just look for the North Star and decide.
