Proud Teacher

Three students from my WOM, Buzz, and Viral Marketing Communication class (Summer 2006) will be presenting at the WOMMA Research Symposium in D.C. Here's the press release!

Congrats Carrie, Ken, and Nick!

-->
Tags: WOM word of mouth Word-of-Mouth Marketing buzz marketing viral marketing marketing communication

Does the Morgan & Rego Study in Marketing Science Undermine the Net Promoter Score Metric?

Since my series of blog posts about the Morgan & Rego article in Marketing Science that tested the value of different customer feedback metrics on predicting business performance, I've been asked if this study undermines the Net Promoter Score (NPS) as a useful metric for companies.

First, let me clarify that the following response is not meant to provide an endorsement or detraction regarding the use of the Net Promoter Score metric. Rather, it's to look at the available evidence from the Morgan & Rego study and assess the implications of that study.

OK, with that aside, the short answer is that we don't have sufficient data to come to a conclusion on the matter. As I noted in my series (here, here, and here) the main reason for this is that the Morgan & Rego study does not actually measure the Net Promoter Score in the same way that Reichheld, Satmetrix, and Bain & Co. do, so it's not fair to make a comparison. (This post explains the difference between how the Reichheld/Satmetrix/Bain "Net Promoter Score" is calculated and how the Morgan & Rego "net promoter" metric was calculated for the study).

Other researchers have also noted this disparity. Specifically, I recently learned that a manuscript has been submitted for publication (Keiningham et al., 2006)* whose goal is to correct and clarify the Morgan & Rego study on this point. In that manuscript they explain why the difference in calculation may actually make a difference in the conclusions that Morgan & Rego come to. Specifically, these authors contend that Morgan and Rego appear to have significantly misunderstood the data fields from which they calculated Net Promoter and Number of Recommendations. As a result, Net Promoter and Number of Recommendations were not actually examined. Therefore, conclusions regarding the effectiveness of the Net Promoter metric advocated by Reichheld on business performance cannot be accurately made from the Morgan & Rego study. Because the Keiningham et al. manuscript is under review I cannot provide more detail publicly at this time, but stay tuned!

For more details about what I see as the implications of the Morgan & Rego study on companies invested in understanding the importance of word-of-mouth marketing please read my last post.

* Keiningham, T., Aksoy, L., Cooil, B., & Andreassen, T. W. (2006). Net Promoter, Recommendations, and Business Performance: A Clarification and Correction on Morgan and Rego. Manuscript submitted for publication. (Thank you to these authors for allowing me to cite their paper in this blog post.)


-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

What Are the Implications of the Morgan and Rego Study for Companies Focusing on Word of Mouth Marketing?

Over the weekend I posted a series of entries summarizing an important article that tested the value of various customer feedback metrics to understand how they relate to key indicators of business performance. I reviewed what the authors saw as the implications of this study and now I want to offer some of my thoughts as well.

Let’s start with the use of the Net Promoter Score. If the Morgan & Rego had computed their “net promoter” metric in the same way as the Bain/Satmetrix “Net Promoter Score” and then found the same results, then this study is a very big deal because a compelling reason companies have adopted the NPS metric is because of its correlation with revenue growth. We’ll have to await future studies from these or other authors that make an apple-to-apple comparison.

I haven't heard a response from Fred Reichheld or Bain/Satmetrix, but I imagine their response might be similar to a recent blog post Fred Reichheld made on his blog.

In response to other criticisms of the Net Promoter Score (besides Morgan and Rego), Fred Reichheld makes two points in the metric’s defense. First, he argues that NPS was never based on statistical correlations but instead based on the relationship between customers’ survey scores (their likelihood to recommend to others) and their subsequent behavior. He states that “People who rate a company higher on the NPS scale buy more and refer more friends to the company than people who rate it lower” (though Morgan & Rego would also dispute this; see p. 437). Reichheld claims that the statistical correlations are “not the foundation of the NPS theory, merely supporting evidence.” According to this reasoning, then, the Morgan & Rego finding showing a non-significant correlation for the net promoter metric and business performance is far less damaging.

The second defense that Riechheld offers is that industry definition is extremely important. He says that you can’t just look for correlations between NPS and business performance in the “online retailing” industry because this is too broad (for example, Home Depot and Victoria’s Secret are both in retailing, and LandsEnd.com and Brookstone.com are both online retailers, but neither of these pairs are really in the same business.

Reichheld’s bottom line is this: “In testing the relevance of NPS to your business, avoid starting with correlations. Instead, begin with real behaviors of individual customers over time. Then, when you examine the correlations of NPS and growth rates, focus your analysis on your true competitors.” This point about industry specificity is crucial since one of the limitations that Morgan & Rego identify is that they can generalize across industries but cannot account for differences within or between industries (see p. 437).

From my perspective, even if companies find value in using the NPS, the prescription that it’s the “only” number a company needs to grow is misleading (many others have noted this as well). For example, if a company has a low NPS score it’s important to understand *why* the number is low so that the company can improve. Thus additional research to determine why people talk positively or negatively is still necessary. Additionally, companies need to focus on other business indicators besides just revenue growth (therefore Morgan & Rego’s critique that revenue growth is not the only indicator of business performance is still valid).

So, should a company who’s currently using the Net Promoter Score stop using it based on the findings from the Morgan & Rego study? For many companies, it seems that it has helped the company focus on creating a better product or service experience for the customer. It has also directed people’s attention to the importance of customer word or mouth.

If a company is not currently using the Net Promoter Score, should they start using it? My best advice here is to let the power of WOM loose and seek out those companies who have used NPS to find out about their experiences in their particular industry. Let the power of a peer recommendation work its magic.

OK, so setting aside the NPS, Morgan and Rego’s study is also relevant to the WOM marketing industry because it confirms the value of ensuring customers have satisfactory experiences such that they don’t generate negative WOM or complaints. Additionally, companies need to focus on customer complaints, use these as a source of consumer insight into what makes them satisfied, and also better handle the complaints to offset the deleterious effects of negative WOM.

I’d love to learn what readers think. What do you think are the implications of the Morgan & Rego study for the WOM marketing industry?

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

Morgan & Rego Study: Limitations & Avenues for Future Research

Limitations

First, the findings are not generalizable to smaller firms or B2B businesses since these weren’t included in the study.

Second, it's difficult to control for differences between industries, and since there may be differences in the preconditions leading to loyalty in each industry, this limitation may affect the utility of satisfaction and loyalty metrics.

Third, all customers were treated equally in the analysis. There's no data on which consumers are most relevant to a firm's success (for example, where should a firm spend the majority of its resources, on the best customers, on the most number of customers, those who complain the most, etc.)?

Fourth, the study was limited to customer feedback mechanisms that were easy for employees and managers to comprehend. Further, the relationship between these and a firm's performance was linear; non-linear relationships and interaction effects among the customer feedback variables might be more useful. NOTE: To understand the importance of how non-linear and asymmetrical relationships might be relevant read this article by Anderson & Mittal (2000) [opens into PDF].

ALSO NOTE: As mentioned before the way the net promoter score was calculated in this study is different than the Bain/Satmetrix Net Promoter Score. Stay tuned, however, from these authors for using the same language and method in future studies.

Avenues for Future Research

First, maybe the reason that satisfaction is more related to firm performance is becuase it costs more for the firm to generate positive WOM recommendations than for customers to simply be satisfied.

Second, promoters don't seem to buy more nor does their influence on potential new prospects seem to be as strong as people believe. Why? Maybe the process of getting customers to engage in positive WOM paradoxically increases their involvement in the category and their desire to seek out the variety offered by other brands for future purchases. Or maybe people who engage in positive WOM are more likely to be opinion leaders who find utility in seeking out variety in brands and companies. Thus, more research needs to be done about the impact of recommendation behaviors on not only the behaviors of others, but also the behaviors of the person doing the recommending. Also, the authors wonder if more active repurchase behaviors that indicate loyalty, like share of wallet, are better predictors of financial performance than the self-reported attitudinal indicators of loyalty which tend to be more passive.

Third, some of the correlational findings between recommendation and satisfaction measures seem to contradict service-profit chain logic (opens into PDF). There was a significant positive correlation between number of recommendations and the proportion of customers complaining. The authors wonder to what extent are WOM behaviors, both positive and negative, driven by consumer characteristics versus the firm’s marketing actions?

So, in conclusion, then, the authors maintain that their results show the value of customer feedback metrics and their ability to predict business performance of the firm. Further, they argue that the best feedback metrics are average customer satisfaction, Top 2 Box customer satisfaction scores, proportion of customers complaining, and the repurchase likelihood loyalty metric. The authors failed to find support for the predictive value of loyalty metrics based on data from recommendation behaviors, net promoters, and the number of recommendations.

This is the end of the Morgan & Rego article summary series. Here are some additional thoughts on the implications of this study.

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

Morgan & Rego Study: Discussion & Implications

Discussion & Implications

Why is the study important? Here are the factors that the researchers identify. First, the authors link customer feedback measures with previously unexplored measures of financial performance, like Total Shareholder Return and Sales Growth. Their findings that Average Customer Satisfaction Scores and Top 2 Box Customer Satisfaction Scores actually do predict sales growth directly counter Reichheld’s claim that they do not. Their findings also counter previous findings that found no relationship between certain variables (for example, average customer satisfaction and gross margins were not found to be significantly related in past research but were found to be so in this research). There is also a positive relationship between customer satisfaction and market share. All of this is to say that a firm’s ability to satisfy its customers has an important impact on that firm’s business performance.

Second, this study shows the impact of customer complaining behavior on business performance. Previous research suggested that firms should try to actually increase the number of customer complaints so that the concerns of these “at-risk” customers can be better addressed. While there was one positive relationship between complaining behavior and market share (that is, more complaining behavior, more market share) the results of this study suggest that these complaints have not been adequately “heard” by the company, or if they have been “heard,” then the firm’s attempts to address the concerns have not counter-acted the negative effects of the customers’ complaining behavior on subsequent business performance. NOTE: The positive relationship between market share and complaining behavior is not surprising given Robert East’s work that suggests companies with higher market share tend to have both more positive AND negative WOM.

Related to this point about complaining behavior, the authors state that existing research by TARP (1986) suggests that customer complaints aren’t a good indicator of customer satisfaction but the authors state that the results of this study suggests that monitoring customer complaints provide insights into customer satisfaction and is valuable for predicting future business performance (the evidence for this is that the customer complaining variable and the other two satisfaction metrics were correlated with one another, and further there were similar patterns in the regressions across all three satisfaction measures).

Third, the study sheds new light on the relationship between customer loyalty and business performance. First, this study found that repurchase likelihood is related to a firm’s business performance which increases confidence in the current managerial practice of paying attention to repurchase likelihood. Second, the study also sheds light on the significance of positive consumer recommendations. In short, the authors argue that focusing on likelihood to recommend is not as useful as focusing on repurchase likelihood. (Other authors argue that you need to focus on recommendation likelihood because repurchase likelihood gets confused by inertia, indifference, or exit barriers that the company puts in the way to make it harder for customers to switch brands [see Reichheld, 2003, p. 48]).

The good news is that customer feedback systems can help a firm implement planning and control measures as there are metrics that help a company to predict business performance. Which ones to use, though, then becomes the issue. This study finds that the three customer satisfaction metrics and the repurchase likelihood metric (loyalty) are the best ones. The average number of recommendations seems only to have a positive impact on future market share and have a negative impact on future gross margins. The authors contend that the net promoter metric seems to have no predictive value at all. The data from this study suggests that increasing the number of promoters will not help the company’s business performance. Rather than focusing just on the net promoter score, companies should focus on a “scorecard” method that includes the following four metrics: average customer satisfaction, Top 2 Box satisfaction, proportion of customers complaining, and repurchase intent.

OK, now we'll discuss the limitations of this study, future avenues for research, and the researchers' conclusions.

---
Reichheld, Fredrick E. 2003. The one number you need to grow. Harvard Business Review (December) 46–54.

TARP. 1986. Consumer Complaint Handling in America: An Update Study. Technical Assistance Research Programs, White House Office of Consumer Affairs, Washington, D.C.

*** Information in this post adapted from Morgan, N. & Rego, L. Marketing Science, Vol. 25, No. 5, September–October 2006, pp. 426–439.

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

Morgan & Rego Study: Results

Results

The researchers found that firm and industry characteristics did have an effect on financial performance (so it was good that the authors controlled for them!).

Above and beyond these characteristics then, the six behavioral measures also explained additional variance, from a low of 1% (for Total Shareholder Return) to a high of 16% (for market share).

Below is how well the each of the six customer satisfaction and loyalty metric explained the variance in the six business performance measures. The customer feedback metrics are listed in the order of their predictive power.

- Average Customer Satisfaction Score. This metric was statistically significant across all six business performance measures. It explained from 5% (net operating cash flow) to 16% (market share) of the variance in these performance measures.

- Top 2 Box Customer Satisfaction Score. This was statistically significant across 5 of the 6 performance measures and approached significance on the 6th one as well (Total Shareholder Return). It explained from 5% (net operating cash flow) to 16% (market share) of the variance in these performance measures.

- Proportion of Customers Complaining. This was statistically significant across 4 of the 6 performance measures (except Total Shareholder Return and Net Operating Cash Flow). It explained 4% (TSR) to 13% (market share).

- Repurchase Likelihood. This was statistically significant across 4 of the 6 performance measures (except Total Shareholder Return and Net Operating Cash Flow). It explained 4% (TSR) to 15% (market share).

- Number of Recommendations. This was statistically significant only across 2 of the 6 performance measures (Gross Margin & Profit Share). But the gross margin coefficient is negative! It explained 1% (TSR) to 12% (Tobin’s Q).

- Net Promoters. This was not statistically significant across any of the 6 performance measures… none! It explained from 2% (market share) to 12% (sales growth) of the variance in these performance measures.

OK, let's move to the discussion and implications of these results.

*** Information in this post adapted from Morgan, N. & Rego, L. Marketing Science, Vol. 25, No. 5, September–October 2006, pp. 426–439.

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

Morgan & Rego Study: Purpose and Methods for Study

OK, what follows are excerpts of my notes from reading this article.

The content was adapted from the article and I generally indicate with quotation marks where I've pulled direct quotes, but because these are from my informal notes I often cut-and-pasted from the article.

Purpose of Study

To determine which of the available customer feedback systems currently used by practicing managers (based on measures of customer satisfaction and loyalty) best predict a company's financial performance.

Method

Data Set

The authors took data from the American Consumer Satisfaction Index (ACSI; provided by the National Quality Research Center at the University of Michigan), which closely matches satisfaction and loyalty data that companies would have available in their own customer feedback mechanisms. Since 1994, the ASCI has collected data from 50,000 consumers annually from 200 of the Fortune 500 companies from 40 different industries to measure consumer evaluations of these companies’ products and services. Utility companies were removed from the analysis because of their monopoly position and from private companies since their financial data was not available. 80 different companies were represented for 7 years (1994-2000).

Financial Performance Measures

There were 6 measures of financial performance:

- Tobin's Q. Compares a firm’s market value to the replacement cost of its assets; forward-looking financial market measure

- Net Operating Cash Flow. Measures a firm’s ability to generate cash; historical accrual accounting info-based measure

- Total Shareholder Returns. Measures firm’s ability to deliver value to shareholders by increasing price of firm’s stock and distributing dividends; forward-looking financial market measure

- Sales Growth. Measures increase/decrease in sales revenue; customer market-based measure

- Gross Margin. Ratio of gross profit to sales revenue; shorter-term

- Market Share. Percentage of sales a firm has relative to entire industry sales; customer market-based measure

Customer Feedback Measures

There were 6 measures of customer feedback:

- Average Customer Satisfaction Score (Satisfaction Measure). This is the mean score on the three specific indicators used to estimate the ACSI latent satisfaction index. The three measures are 1) overall satisfaction, 2) expectancy disconfirmation, and 3) performance versus their ideal product or service in the category.

- Top 2 Box Customer Satisfaction Score (Satisfaction Measure). This refers to the two highest-scoring points on the five-point scale that firms typically use to capture customer satisfaction. Because the ACSI uses 10-point satisfaction scales, this metric was operationalized as the proportion of customers surveyed that rated the firm in the top 4 points on the 10-point single-item “overall satisfaction” ACSI scale.

- Proportion of Customers Complaining (Satisfaction Measure). Number of consumers of a firm who voice dissatisfaction with the product or service versus those who do not. This was calculated using the ACSI “voice” variable that comprises two items that ask if the consumer has either formally (as in writing or by phone to the manufacturer or service provider) or informally (as to others) complained about the product or service.

- Net Promoters (Identified as Loyalty-based Measure; could also measure advocacy). Percentage of a firm’s customers who make positive recommendations of the company brand to others minus those who do not (NOTE: this differs from Reichheld’s definition since his refers to a likelihood to make a recommendation).

The net promoter score in this study utilized ACSI data concerning consumer responses to the questions “Have you discussed your experiences with [brand or company x] with anyone?” and “Have you formally or informally complained about your experiences with [brand or company x]?” The first question measures both positive and negative recommendations, while the second question measures negative recommendations. Net promoters computed as the number of a firm’s surveyed customers that reported discussing their consumption experiences minus the number of the firm’s surveyed customers that reported formally or informally complaining expressed as a proportion of the total number of a firm’s surveyed customers.

- Repurchase Likelihood (Loyalty-based Measure). Customer’s stated probability of purchasing from the same product or service provider in the future. This was taken from the ACSI that asks consumers to rate “How likely are you to repurchase this brand/company?”

- Number of Recommendations (Loyalty-based Measure; could also measure advocacy). This refers to the number of people to whom consumers of a firm’s product or service engaged in positive word-of-mouth (WOM) behavior as captured in the authors' net promoters variable report having recommended the brand or company. The ACSI question asks: "With how many people have you discussed [brand or company x]?” Authors averaged this metric at the firm level, representing the average number of people to whom the surveyed customers of a firm who engaged in positive WOM have recommended the brand or company.

Co-Variates and Other Industry Characteristics

To account for effects of other factors the authors identified a number of firm-level and industry-level co-variates: for the firm, number of business segments the firm competes in, the intensity of advertising and R&D expenses, and size of assets; for the industry-level, Hirschmann-Herfindahl index (HHI; because market structure can affect financial performance) and demand growth.

To control for other industry characteristics, “we included three dummy variables in our analyses: ACSI sector definitions to identify service-focused versus physical goods-focused firms, annual reports to identify firms that market direct to their end-user consumers versus those using intermediaries, and the ASCI survey data collection protocol to indicate firms that face longer versus shorter interpurchase cycles.”

OK, so basically the authors are going to take the six customer feedback metrics (3 satisfaction and 3 loyalty metrics) and see which of these best predict the six business performance measures.

We should note at this point that the "net promoter" metric in this study is not calculated in the same way as the Net Promoter Score, which is calculated by subtracting the number of detractors (0-6 on a 10-point scale indicating how likely a person is to recommend a company or brand to others) from the number of promoters (9-10), disregarding the number of passive responses (7-8).

Next, the results!

*** Information in this post adapted from Morgan, N. & Rego, L. Marketing Science, Vol. 25, No. 5, September–October 2006, pp. 426–439.

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

Very Important Study! -- The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance

OK, this is a big one.

There’s a very important study for those interested in WOM marketing that has been published in Marketing Science, a well respected marketing journal, about which customer satisfaction and loyalty metrics best predict a firm’s business performance. It’s an even bigger study for those invested in the use of the Net Promoter Score.

Much has been made of the Net Promoter Score (NPS) as a powerful metric for companies to get a handle on the WOM activity of consumers. According to the proponents of the NPS, asking one simple question, how likely a person is to recommend a company or brand to their friends or colleagues, will allow you to effectively monitor a firm's performance (e.g., a firm's level of revenue growth). They claim that other metrics, such as satisfaction measures that so many firms routinely use, are far less important and perhaps not even worth using. While Reichheld and colleagues have been praised for coming up with an easy-to-implement solution to benchmark their success with WOM, they have also come under fire for over-simplifying the process of WOM tracking by advocating for just this one question.

At least one study has been conducted to confirm the findings of the NPS, extending it from U.S. companies to U.K. companies (opens into PDF file; see my blog post about this study), but there has been no peer-reviewed academic journal articles where the central premises of the NPS have been studied. Until now.

Researchers Neil Morgan and Lopo Leotto Rego entitled their article "The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance."

In their manuscript they offer what amounts to a scathing critique of the NPS arguing that it's 1) ineffective in predicting business performance and 2) misguided for companies to simply use this metric. By the way, these same researchers published counter-point letters-to-the-editor in the Harvard Business Review following the publication of Reichheld's influential article in the same publication.

This article deserves a careful read and consideration. However, like any research, there are important limitations that we also need to consider. Over the next few posts I'll offer a summary of the article and what I see as its implications to the use of the Net Promoter Score by companies and the WOM marketing industry.

For starters, here's the abstract (due to copyright restrictions I won't post the article here; it can be accessed at your local university library*):

Managers commonly use customer feedback data to set goals and monitor performance on metrics such as “Top 2 Box” customer satisfaction scores and intention-to-repurchase” loyalty scores. However, analysts have advocated a number of different customer feedback metrics including average customer satisfaction scores and the number of “net promoters” among a firm’s customers. We empirically examine which commonly used and widely advocated customer feedback metrics are most valuable in predicting future business performance. Using American Customer Satisfaction Index data, we assess the linkages between six different satisfaction and loyalty metrics and COMPUSTAT and CRSP data-based measures of different dimensions of firms’ business performance over the period 1994–2000. Our results indicate that average satisfaction scores have the greatest value in predicting future business performance and that Top 2 Box satisfaction scores also have good predictive value. We also find that while repurchase likelihood and proportion of customers complaining have some predictive value depending on the specific dimension of business performance, metrics based on recommendation intentions (net promoters) and behavior (average number of recommendations) have little or no predictive value. Our results clearly indicate that recent prescriptions to focus customer feedback systems and metrics solely on customers’ recommendation intentions and behaviors are misguided.

Very exciting stuff! Let's dig in...

* You can find the article in Marketing Science, Vol. 25, No. 5, September–October 2006, pp. 426–439.

UPDATE: You can find a pre-press version of the article on Dr. Rego's site. [Thanks to Constantin Basturea for pointing this out!]

-->
Tags: Net Promoter Score word of mouth Word-of-Mouth Marketing WOM Loyalty marketing communication

Blogging Success Study Released!

It's been a long time in the works, but finally released!

John Cass (Backbone Media) and I conducted a research study with my class in Advanced Organzational Communication (Spring 2006) on what makes for a successful blog. We identified a number of best practices and also five themes that cut across all the interviews:

• Culture: If a company has particular cultural traits worth revealing or a bad reputation it wants to repudiate, blogging can be an attractive option.
• Transparency: Critical to establishing credibility and trust with an audience. People want to see an honest portrayal of a company and know that there are not ulterior motives behind the blog. Blog audiences respect a willingness to disclose all points of view on a subject.
• Time: It takes a lot of time to set up, research and write a quality blog. Companies need to identify a person who has the time or whose schedule is freed up to make the time, or need to engage a group of people to share the responsibility.
• Dialogue: A company’s ability and willingness to engage in a dialogue with their customer base about topics that the customer base is interested in is critical to its blogging success.
• Entertaining writing style and personalization: A blogger’s writing style and how much they are willing to reveal about their life, experience and opinions brings human interest to a blog, helps build a personal connection with readers and will keep people reading.

You can download the study as a PDF (name/e-mail registration required) or read it online. The online report takes the form of a blog so that people can comment on individual sections and is hosted on the Scout blogging server.

Be sure to check out the summaries of the interviews with each blogger to learn more detail about each blogger's experience.

Thanks so much to all the bloggers who agreed to participate in the study, the students in my class who did the interviews, and the folks from Backbone Media (John Cass, Stephen Turcotte, Megan Dickinson, Kristine Munroe and Dave Alpert).

Northeastern University Press Release

-->
Tags: blogs blog study Northeastern University Backbone Media