Net Promoter Scores
July 7, 2005 at 8:22 pm #39936
SandraMember@Sandra Include @Sandra in your post and this person will
be notified via email.
Hi everybody,I was wondering, is there any of you using the Net Promoter Score developed by Reichheld? If so, do you have any idea of what the minimum sample size should be in order to have an accurate estimate? Do you know if this metric has any drawbacks? ThanksSandra0July 22, 2005 at 10:04 pm #123510
SunrayMember@Sunray Include @Sunray in your post and this person will
be notified via email.
Sandra, I just heard that a major US Corporation is adopting this measure, and I was astounded that they would place so much weight behind it. I wrote an article almost three years ago pointing out the tremendous weaknesses this measure holds. I admire Reichheld as a consultant and thinker, but he just doesn’t know anything about measurement, particularly what determines ROI short or long term.
Not sure if you are able to contact me for my review, but bottom line:
1. NetPromoter score is based on pseudo correlational analysis, the difference between those who identify themselves as promoting (7-10 on a scale) vs. non promoting (0-6 on same scale). Well right away when you dichotomize a ‘scale’, you are discarding extremely important information and moving the level of measurement down a notch. So, now your ability to ‘correlate’ to long term ‘equity’ is starting to get pretty clouded. You also destroy the use of predicitive analytics, which seems at cross purposes with what you are trying to achieve.
2. Correlation is not causation, and that is how Riechheld’s method is purported to work – Netpromoter effect = long term profitable customer. As far as I can see, there are no controls for using this technique- no baseline group, except through the use of weak correlational design (estimated or synthetic baseline), and no validity checks that I can see. What you get is a ‘reliable’ indicator, in that you can use it over and over and get a result, that could be totally off the mark (invalid) – Most people don’t go to the next step which is to calibrate to real dollars (incremental dollars over time) to see if indeed this correlation is holding up. They stop with the Netpromoter score ‘as if’ it is the actual measure, when instead it is a proxy.
There is more technical stuff I’d love to share.
My observation is that a sample size of between 500-1500 (if you want to project to a national sample, for example) will achieve an accurate Netpromoter result (Minimum N of 50, sampled randomly, of course). I am questioning, however, whether the measure means anything at all!
If you are into Six Sigma, there are much better ways to do this, with some of the more advanced DOE or even more simple test/control designs. This is the stuff that agriculturalists, manufacturers, and operations folks have been using successfully for years. But you need a more carefully design experiment to get at net effects of ‘loyalty’.
Sorry if this is harsh, but it really bugs me that this stuff like netPromoter is being used for such important purposes.0
The forum ‘General’ is closed to new topics and replies.