Apple-to-Apple Comparisons for WOM White Paper

John Moore recently provided a follow-up post to his comments on my collaborative research project with BzzAgent and the results reported in the white paper entitled "The Value of Managed WOM". In his post he concluded that "the two samples appear too dissimilar to draw any empirical conclusions" regarding frequency data for the number of interactions, WOM episodes, and episode-to-interaction ratio (the percentage of a person's total interactions that include a WOM episode). Below is my comment that justifies why I think it is an apple-to-apple comparison.

***
Hi John,

I enjoyed reading your follow-up post about my WOM research project with BzzAgent. However I want to clear up a misunderstanding in your post. I did provide the additional demographic information on the BzzAgent sample, which was as you reported (though see next paragraph). However, in order to be able to make apple-to-apple comparisons I matched the BzzAgent sample to the "everyday people" convenience sample by age and education level. That is, from the total BzzAgent sample I created a sub-set of those agents with "Some College" and aged 18-29. This is reported in the white paper on Footnote #1 on page 3.

You are right in pointing out that there is a higher representation of females in the BzzAgent sub-set that I used to compare with the convenience sample (it's actually 88% [BzzAgent sub-set] to 74% [convenience sample] for the variables under consideration; let me know if you want more details about how these perecentages were determined). However, among both the BzzAgent sub-set and the everyday people convenience sample there were no statistically significant differences for the number of interactions, number of WOM episodes, and the episode-to-interaction ratio (the percentage of total interactions that included a WOM episode) based on sex of the respondent.

Therefore, I think one can safely conclude that the BzzAgent sub-set and the everyday people convenience sample is an apple-to-apple comparison based on education level, age, and sex.

Thanks again for your interest in this research!

Walter Carl
***

I would also note to John, and other interested readers, that a larger sample of "everyday people" (that is, those not affiliated with a word-of-mouth marketing agency) is warranted and anyone interested in helping to fund that study should contact me... everything is ready to go :-)

Response to John Moore's (BrandAutopsy) Comments

Hi John,

Thank you for your comment and questions regarding my "What's All The Buzz About?" article! I thought I'd reply as a new post so that I can provide a more detailed response. I'll intersperse my comments within yours...

***

Your report says managed word-of-mouth (WOM) programs, like those from BzzAgent, generate more WOM than happens from everyday people. Interesting, but not a surprising finding especially considering BzzAgents are most likely more skillful at recognizing and reporting WOM activity than their peers in your convenience sample. What are your thoughts about BzzAgents potentially being more skillful at knowing when and where WOM happens and thus being able to report such activity for your study?

***

This is an interesting point. First, I agree with you, it shouldn't be a surprising finding that BzzAgents report more WOM episodes than non-Agents. The amount of the difference was what surprised me (for example, Agents having twice as many of their total interactions include a WOM episode). Second, I don't have any direct measurement of whether or not BzzAgents are more or less skillful at recognizing WOM opportunities. My sense is, however, and like yours, that they probably are better able to recognize WOM opportunities. The point I think you're getting at, though, is were they this way "naturally" or does it have something to do with BzzAgent. I don't know, but here are my thoughts:

First, at the very least, I think Agents are more conscious of, and attuned to, the WOM process. I think the more a person's awareness of WOM is raised, and the more a person is encouraged to spread WOM, then the greater likelihood that person will see opportunities to do so (it's kind of like the social-psychological process of buying a certain kind of car and then "suddenly" noticing how many other cars like yours are on the road).

But more importantly, another reason why BzzAgents might develop better acuity in seeing WOM opportunities is that there is a social system in place that a) provides positive feedback about the WOM activity and b) creates a community where this activity is supported. As Dave Balter and Matt McGlinn have talked about at WOMMA conferences, when BzzAgents submit reports, they receive personalized responses from company representatives about their WOM activity. This interaction between the Agent in the field, and the person at the "Central Hive", makes their Agents' own WOM activity part of a larger social process of participation in a community, which creates more identification with the values of that community, and one of those values is to share your feelings about ideas, brands, products, and services with other people (that is, engaging in WOM).

So, whether the BzzAgent system helps people to be better at recognizing opportunities for WOM, or the Agents were like that already (which may have attracted them to BzzAgent) is an open question. Either way, I would wager that any WOM program that makes engaging WOM part of a larger social and relational process, whether it's BzzAgent or some other model, will lead to increased levels of WOM activity. The non-Agents in the study don't have that added sense of being part of a larger community of passionate people spreading WOM.

And of course, BzzAgents have more opportunity for participating in WOM when they do so as part of a buzz marketing campaign. But very interestingly, and I think this is one of the more fascinating results, the majority of the BzzAgents' reported WOM episodes were not related to buzz marketing campaigns. But instead they were "everyday" WOM unrelated to specific products with BzzAgent. I have a whole section on the implications of this finding in my paper and why clients of word-of-mouth marketing agencies shouldn't want Agents to buzz about their product 24-7 (see my last comment at the end of the post for more details).

***

And ... help me to understand how similar, in demographics and psychographics, your two sample sets are/were. The white paper only outlines the everyday, non-BzzAgent sample as being college educated adults ranging in age from 18 to 29. Since the paper empirically states the BzzAgent sample set is more socially active, I’d like to know if BzzAgents are potentially more socially inclined than are the people in the everyday, convenience sample.

***

Another interesting question. I didn't have access to specific psychographic data on the BzzAgents, or the non-Agents, so I can't provide that detail for you. Additional demographic data was available for the Agent sample, and it's reported in my article. Here's an excerpt from the methods section:

"The participant sample was representative of the larger Agent population on all available demographic variables: sex, age, ethnicity, education level and income levels. 83% of the sample was female; ages ranged from 13-72 with 45% being 18-29; approximately 90% self-identified as white or European-American; 54% had some college or a 4-year college degree; and approximately 65% reported incomes within the range of $20,000 - $80,000 per year (26% reporting over $80K and 9% reporting under $20K)" (from Carl, in press).

For the convenience sample, the only demographic information collected was sex, age, and education level. 67% of the sample was female and the average age was right around 20. All were pursuing a 4-year college degree.

Now, the bit about if they are more socially inclined. Do you mean are they more introverted or extroverted, to use trait-based psychological terms? I don't have measurements of that, but Agents certainly reported engaging in more interactions on average. Specifically, about 140 interactions over a 7-day period for Agents versus 108 for non-Agents. (An interaction was defined as any conversation that a person considered meaningful and that was more than just a "hi and bye" made in passing). Thus, the white paper reports engaging in more interactions as an indication of being more "socially active," but we don't have data on whether or not they are more socially "inclined." I wouldn't want to speculate much more beyond that., though I'd be interested to hear your thoughts on that point.

I'll have new data coming in shortly for non-Agents about frequency of interactions, WOM episodes, and the E/I ratio (that is, the percentage of one's total interactions that include a WOM episode). I'll report those statistics when I get the data cleaned and analyzed.

I'd also like to respond to "Paughnee" who posted this comment to your blog:

***

Most BzzAgents are probably in the habit of making a mental note of WOM in a conversation whether they are reporting it or not.

***

Perhaps, but both Agents and non-Agents had similar instructions and a similar log sheet about what to record and what not to record. So I don't think this would be a big factor in how they reported it on the "Total Interactions and Word-of-Mouth Episode Worksheet" (the original log sheet I designed for this study; e-mail me and I can send you a copy if you're interested).

***

Thanks in advance for helping this WOM evangelist become more enlightened.
johnmoore (from Brand Autopsy) Homepage 08.04.05 - 12:28 pm #

***

My pleasure! Thank you for your interest in my research. I'd love to continue this conversation further! :-) I can send you a pre-press version of the full article if you're interested (just e-mail me). I would just ask that it not be posted or distributed as there are copyright issues based on the contract with the publisher for the academic journal.

Cheers :-)

Walter

--- Relevant Links ---

- BzzAgent Blog Post: "The Value of Managed Word-of-Mouth"
- BzzAgent White Paper: "The Value of Managed Word-of-Mouth" (PDF)
- Brand Autopsy's Blog Post: "Thoughts on a WOM White Paper"
- Church of the Customer Blog Post: "The Science of Word-of-Mouth"

---



This work is licensed under a Creative Commons License.

"What's All The Buzz About?"

I recently learned that my manuscript entitled "What's All The Buzz About? Everyday Communication and the Relational Basis of Word-of-Mouth and Buzz Marketing Practices" was accepted for publication in Management Communication Quarterly (to appear in Volume 19, 2006, Sage Publications). Here's the abstract for the article:

Although the influential role of word-of-mouth communication has been known for decades, a more recent phenomenon is the emergence of explicit organizational efforts to stimulate “buzz” (contagious talk about a brand, service, product, or idea). Buzz marketing organizations either pay people, or seek volunteers, to try new brands and then have agents “talk them up” among their social networks. This manuscript contends that the practice of buzz marketing is usefully framed within the study of everyday communication, which has shown considerable growth in the field of communication studies over the past decade. This framework provides a reliable scientific source and basis upon which to found further claims about the effectiveness and ethics of buzz marketing.

The study sought answers to questions of frequency of word-of-mouth (for example, what percentage of the total number of interactions includes word-of-mouth about an organization, brand, product, or service), what medium (such as face-to-face, phone, or online) was most commonly used, how common recommendations or referrals were, the conversational context surrounding a word-of-mouth episode, and what percentage of buzz marketing agents’ word-of-mouth communication is part of a buzz marketing campaign. I also compared the buzz marketing agent data to a "convenience sample" of non-Agents (college-educated 18-29 year olds).

The company with whom I collaborated on the project, BzzAgent, Inc., recently released on their blog a white paper entitled "The Value of Managed WOM Programs" based on some of the findings reported in the article.

A future manuscript (currently being submitted for review to another journal) will report findings from the same study about the quality of the interactions, levels of conversational control, value to people's lives, impact of the word-of-mouth episodes to people's thoughts, feelings, attitudes, and behaviors, as well as any changes in people's relationships and levels of attraction. Each of these factors (quality, control, value, impact, and relational change) were analyzed according to sex of the agent and their conversational partners, the relationship between the two (stranger, acquaintance, friend, best friend, relative, etc.), and also whether the episode was institutional word-of-mouth (that is, part of a buzz marketing campaign) or everyday WOM (unrelated to a campaign).

Addendum (September 2005): A pre-press version of this manuscript can be accessed from my Northeastern download page (you will be greeted with a simple form to fill out that asks for your name, institution, and e-mail address). The official version of the manuscript will be published in Management Communication Quarterly (2006). Feel free to comment on the piece should you desire to do so but please provide the link to my download page (http://www.atsweb.neu.edu/w.carl/downloads/) rather than uploading or mirroring the PDF file.

---



This work is licensed under a Creative Commons License.