"One Question, and Plenty of Debate": More Scrutiny of the Net Promoter Score

Readers of my blog know I've been following the recent academic research seeking to establish a relationship between likelihood to recommend metrics and business performance measures. In these articles the Net Promoter Score (NPS) has come under intense scrutiny.

The first study was the Morgan & Rego study published in Marketing Science (see series of posts beginning with this one for a summary). This is a very interesting study to read but a significant limitation was that it did not calculate the NPS as the creators do (Fred Reichheld, Bain, and Satmetrix), thus it wasn't an apples-to-apples comparison. This same limitation was noted by other researchers as well (see this post for details) and a paper has been submitted to correct and clarify this point (Keiningham, Aksoy, Cooil, and Andreassen, 2006; see Footnote 1 below for citation). Interestingly, this corrected study was also referenced on Fred Reichheld's Net Promoter blog.

But now it's Monday, and Monday's the day for big news to come out, so here we go... In today's Wall Street Journal Scott Thurm reports in his article "One Question, and Plenty of Debate" (subscription required) about a new study that calls into question the relationship between the NPS and business performance. The study was written by the same authors who penned the correction and clarification article in Marketing Science (Keiningham et al.) and it will be published next year in the Journal of Marketing, another very well-respected marketing journal.

According to the WSJ article, this new study was unable to replicate the claims made by Mr. Reichheld, Bain, and Satmetrix that "net promoter scores are better indicators of revenue growth than other customer-satisfaction measures." Further, for two of the industries (airlines and personal computers) cited in Mr. Reichheld's book (The Ultimate Question), the researchers found that a different customer-satisfaction measure, rather than NPS, better explains revenue growth.

Mr. Reichheld is quoted in the article making the point that too much is made of the correlation between NPS and revenue growth and that focusing on this correlation is missing the "forest for the trees." Instead he states that NPS is effective "because it forces top executives, and other managers, to focus on creating happy customers" (this quote was parahrased by the journalist in the article).

The WSJ article goes on to say that this is a consequential issue because many executives are making managerial decisions based on the use of the NPS metric and that critics "are trying to protect executives racing to adopt the net-promoter metric," expressing concern that "the corporate boardroom is probably misinterpreting the importance of this" (the latter quotation is from V. Kumar, a marketing professor at U of Connecticut School of Business, who will also be publishing a study soon that questions the link between NPS and business performance).

The new study by Keiningham et al. entitled "A Longitudinal Examination of 'Net Promoter' on Firm Revenue Growth" is currently embargoed (except for citation in academic journals) until its publication in 2007. However, through correspondence with the first author, I have learned that interested readers can download an executive summary here. The executive summary provides an overview and gives sufficent detail to illustrate that the methodology used was much closer to the methodology used by Mr. Reichheld, Bain, and Satmetrix.

I'll have more to say on the executive summary in a future blog post as well as what this latest research finding means for companies and the WOM industry. I'll also be referencing this debate on the NPS, among many other metrics and research traditions, in my "State of Word of Mouth Research and Measurement" presentation at the WOMMA Research Symposium on Dec. 11th in Washington, D.C.

Disclosure: I was interviewed for this Wall Street Journal article and I am on the Advisory Board for the Word of Mouth Marketing Association.

1. Keiningham, T., Aksoy, L., Cooil, B., & Andreassen, T. W. (2006). Net Promoter, Recommendations, and Business Performance: A Clarification and Correction on Morgan and Rego. Manuscript submitted for publication.

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication