By asking a simple single question, you will predict the potential growth of your organization. It was the promise of a consultant named Fred Reichheld, and the whole business world started to questioning old solid traditions such, the annual customer satisfaction study, the market research ‘old’ world and the need to build scorecards combining different KPIs to control the sanity of the business organizations and monitoring the value creation. It was 2003, I did my first NPS project in 2006. Fred promised, “This number (the NPS) is the one number you need to grow!”. After 11 years and several projects under my belt, can I honestly say Fred has kept his promise?
Fred emphasizes two specific aspects of his kpi: simplicity and depth. My 11 years experience taught me, it is not simple, nor exhaustive. Don’t take me wrong, I strongly believe the whole concept is brilliant, but… it brings with him a long list of ambiguities.
Confusion between the metric and the methodology
The first, and probably worst, ambiguity is the confusion between the metric and the methodology. Calling the metric and the methodology with exactly the same name, it turned out not to be a perfect choice. Like you call EBITDA the processes behind the value creation measured with the EBITDA financial indicator. In very simple terms, EBITDA measures the value creation or destruction. The methodologies to manage the processes measured by EBITDA have absolutely nothing to do with the financial indicator itself. In few words, the confusion was: since NPS is simple to measure, then NPS is easy to implement and govern.
Unfortunately, a proper implementation and a serious governance are two specific activities hard to successfully deploy. A very complex implementation, and a governance that must find a rigorous alignment between the different business units of the organization, and a strong commitment from the c-level suite.
It is not so profound
It turns out is not profound. The NPS itself is not a bulletproof KPI, neither it is the single number you need to grow. The reality stays exactly on the opposite side: explaining the score movements is extremely hard due to the high volatility of the score itself. Moreover, successful organizations implementing the NPS have correlated several other KPIs to it. Again, a very complex and time-consuming process.
Does your customer really recommend?
If your customer is telling you “I’m willing to recommend!”, will he do it in reality? Yet another ambiguity, the willingness to recommend it doesn’t necessarily turn in an action. The best way to check if the customers are going to really recommend your product/service is to use another KPI. For instance, I was working with an online banking, NPS was extremely high. What we decide is to ask all promoters (9-10) “Do you consider XYZ as your primary bank?”. Promoters that answered yes, were really active in promoting, while who answered no, showed a low recommendation behavior. Of course, it means you need to find the proper proxy to proof the transition from the willingness to a real recommendation. Again, time, resources, and money.
It is not the right metric to show trends
The NPS score is perfect to measure improvements over time. In few words, plotting the score on a timeline helps you to understand if you are improving or not. Now, think about a simple situation:
- My team did a very bad job in January and we got from the 26 clients we served a 0. It means 0 NPS Score.
- We worked like hell in February, and we got from 24 clients a set of 6, thus 26 times 6. Our NPS Score didn’t change. Still 0. My boss was furious and my people frustrated.
- I push my team even further in March, got 15 times 6, and 10 times 7. Guss what… my NPS score is still 0, and my boss fired me together with some members of my team.
Of course, I’m exaggerating but look at the same story considering the average. In January my average is 0, on February 6 (Wow! A real improvement of 6 points), and in March again a small improvement +0.4. This is really a good story unfortunately killed by NPS score.
An 11 points scale for a 2 groups score ???
Next ambiguity, an 11 scale is hard to interpret. Do you like your car: yes, no, I don’t know. One of those three answers would likely be easy to pick. Now let’s transform into a 5 Likert scale: do you like your car? Like a lot, Like, neutral, don’t like, definitely don’t like it. A little bit complex, but still manageable. We can also push for 7 points Likert scale, and add more complexity for the respondent, but what about going to an 11 points scale? A lot of noise and confusion. And, moreover, the sad story of using the Score: you are going to reduce the scale to 3 points: detractor, passive, and promoters, eliminate the passive and use just the promoter. Basically, you can simply ask: Would you promote? yes, no, I don’t know and end up exactly at the same result of an NPS Score. Crazy, isn’t it?
Out of the above crazy 11 scale is the issue related to how graphically show the scale (!). I saw some web or email question colored from red to yellow to green. Another sad story: the scale is so complex that you need a visual solution to explain it. You need a red over the 6 to explain to your customers the 6 is bad and has the same influence on the score as the 0 (!). A crazy situation really hard to explain in market research terms.
It tries to measure a future behavior
Next ambiguity, if I ask you about your potential behavior in the next future, what would be the level of precision in your answer? Willingness to recommend is something will happen, probably, in the future. The best research questions are about past behavior, not future behavior. Asking a study participant Will you try to climb mountains? or Are you going to give up cigarettes? requires an ability to predict a future behavior. Our mind works well in what we have done, not in predicting what we are going to do. The final results of a research are also more accurate considering past and actual behavior, not a prediction of what will probably happen in the future.
It doesn’t correlate directly with loyalty and future repurchase
NPS Score strongly correlates with repeat purchases, loyalty, and referrals. Well, later studies show the opposite. Again, this is due to the fact future behavior is really hard to assess. It is something in the future. Something you cannot be sure you are going to do, or not. Asking someone about what is going to do in the future, it is not about the recommendation, loyalty or future repurchase, it is simply about optimism. If we are interested in loyalty, repurchase, or referrals, we can simply ask a customer if he did one of those 3 actions in the past 6 weeks. Or ask new customers if they were influenced by someone in their purchase decision. The question asks about actual past behavior, not about some prediction of future behavior.
It can be used for relational as well transactional measurements
Using the NPS to evaluate a transaction. That is probably the big misusing of the NPS question. So, I have my online banking account and I open a session to execute some payments. Everything goes well and all the payments successfully went through. Now I receive an invitation to give a feedback about my payment session (!). Based on your payment session would you recommend Bank Xyz to a friend or colleague? Wait a minute, are you asking me to recommend your bank based on a transaction I consider, according to the Kano model, as a basic need you need me to satisfy in our business relation? Why would you use NPS for something it was not designed for? Companies using NPS unproperly, find themselves soon in trouble.
It is so easy to analyze unstructured data
Next ambiguity: the real gold is in the open-ended question. And this is absolutely true. But how complex is to build a proper solution to automatically detect topic and sentiment out of those answers? Don’t underestimate that complexity, and remember you need a proper Text Mining platform in place to properly analyze free text. The follow-up question is extremely important and it is the source of the drivers of satisfaction and dissatisfaction. It is one of the beauties of the NPS methodology. Again, you need to implement a proper Data Analytics and Text Mining framework, otherwise, you will probably get frustrated about the outcome.
But Customer Experience still needs a number!
We need a number, we need a number that tells people how we improved Customer Experience. We need a number that explains how good we are in customer centricity. If NPS is not the right number then what is the good one?
Who said that NPS is not a right number, so far we explain the Score itself is not a methodology, neither a good number to track transactional touchpoints. But, if you think to measure the strength of the relationship of your customers, with the counterindications I showed you before, then use it. Simply don’t pretend it is the single number you need to grow because it is not. We cannot simply reduce Customer Experience to a single number, neither not consider the business complexity to manage customer experience.
This is the biggest ambiguity of NPS. It tries to be the universal solution that can’t be universal. It’s loved by c-levels because it looks like something to solve a complex problem in an easy way: the management of Customer Experience that can’t be solved in such simple way.
Of course, you can simply believe in NPS as you can believe in seers, horoscopes or bigfoots. You can also believe in life beyond death but it is just faith, no science.