Monday, November 13, 2006

Does the Morgan & Rego Study in Marketing Science Undermine the Net Promoter Score Metric?

Since my series of blog posts about the Morgan & Rego article in Marketing Science that tested the value of different customer feedback metrics on predicting business performance, I've been asked if this study undermines the Net Promoter Score (NPS) as a useful metric for companies.

First, let me clarify that the following response is not meant to provide an endorsement or detraction regarding the use of the Net Promoter Score metric. Rather, it's to look at the available evidence from the Morgan & Rego study and assess the implications of that study.

OK, with that aside, the short answer is that we don't have sufficient data to come to a conclusion on the matter. As I noted in my series (here, here, and here) the main reason for this is that the Morgan & Rego study does not actually measure the Net Promoter Score in the same way that Reichheld, Satmetrix, and Bain & Co. do, so it's not fair to make a comparison. (This post explains the difference between how the Reichheld/Satmetrix/Bain "Net Promoter Score" is calculated and how the Morgan & Rego "net promoter" metric was calculated for the study).

Other researchers have also noted this disparity. Specifically, I recently learned that a manuscript has been submitted for publication (Keiningham et al., 2006)* whose goal is to correct and clarify the Morgan & Rego study on this point. In that manuscript they explain why the difference in calculation may actually make a difference in the conclusions that Morgan & Rego come to. Specifically, these authors contend that Morgan and Rego appear to have significantly misunderstood the data fields from which they calculated Net Promoter and Number of Recommendations. As a result, Net Promoter and Number of Recommendations were not actually examined. Therefore, conclusions regarding the effectiveness of the Net Promoter metric advocated by Reichheld on business performance cannot be accurately made from the Morgan & Rego study. Because the Keiningham et al. manuscript is under review I cannot provide more detail publicly at this time, but stay tuned!

For more details about what I see as the implications of the Morgan & Rego study on companies invested in understanding the importance of word-of-mouth marketing please read my last post.

* Keiningham, T., Aksoy, L., Cooil, B., & Andreassen, T. W. (2006). Net Promoter, Recommendations, and Business Performance: A Clarification and Correction on Morgan and Rego. Manuscript submitted for publication. (Thank you to these authors for allowing me to cite their paper in this blog post.)