Happy Customers

Happy Customers are Vocal Customers: Benefits and Drawbacks of Social Ratings Websites By: Tim Marco August 20, 2009 http://www.aswenowthink.com [email protected]

Summary This report analyzes reviews from social ratings websites in an effort to uncover consumer motivations to share experiences with others. While some reviews are negative, the distribution of responses clearly skews towards the positive end of the spectrum. Business owners and other interested parties should take note of this tendency and encourage users to write reviews of their companies, which can serve as positive testimonials from prior customers. However, they need to be cautious when using such sites to monitor customer satisfaction, as this bias can lead to falsely positive impressions.

Introduction “I'm sorry that so far all of my reviews are all 5 stars”, Yelp.com user Greg K. wrote in August 2008, “..but I'm still relatively new to Yelp and right now, I'm doing the reviews of my top notch places, people, etc.”1 In the year since, he has written seven additional reviews of local business and restaurants -- six of which were given perfect five-star reviews. The use of social media for marketing and other business purposes is an intriguing topic for business owners. Some, such as authors Clay Shirky2 and Jeff Jarvis3, have

2 offered anecdotal evidence that social media have the potential to pressure companies to provide better goods and services. Because these media can reflect the opinions of a large number of users, the authors argue that companies with poor customer satisfaction will face negative publicity and, eventually, declining revenues. In order for this idea to be correct, these technologies must accurately capture a wide range of consumer opinion. That is, users must be willing to write reviews of goods and services regardless of their impressions. Yet, as will be shown, actual consumers do not tend to behave in this way. Rather, a vast majority of reviewers are more like Greg K., sharing their opinions only for goods and services about which they are passionate. Forrester Research analysts Sucharita Mulpuru, Carrie Johnson and Peter Hult first observed this pattern in a 2007 report4. Analyzing reviews from Amazon.com, they discovered that over 80% of all responses were positive. While their report focused specifically on the retail sector, the analysts stated that they “…believe[d] this information is applicable across a wide variety of verticals – from travel to auto, financial services, or healthcare.” In this study, I reviewed data from some of those other verticals. To learn more about user behavior on social ratings websites, I collected data from over 19,000 reviews across several sites. These sites were: Yelp.com (Chicago, IL microsite), a site that specializes on reviews for local businesses; Metacritic.com, a reviews aggregator for films, books, television, and video games; and allMSU.com, a professor ratings website designed for college students. The variety of sites was purposefully chosen, as the intent is to illustrate that the tendency of positivity occurs across individual sites, users, and verticals. Rather, the trends outlined here and in the Forrester report appear to be universal. The results are clear: a vast majority of the time, users act quite a bit like Greg K. Reviews are overwhelmingly positive, suggesting that consumers are motivated to share positive experiences, yet unlikely to talk about negative ones.

Findings In order to describe user behavior, I compiled aggregate ratings from each site. Because each of these sites uses different ratings systems, the data need to be viewed on a site-by-site basis. The following standards were use to categorize the data: ! ! !

Yelp – Yelp uses a familiar 1-5 star rating system. I consider a ‘positive’ rating on Yelp to be 4 or 5 stars, and a ‘perfect’ rating to be 5 stars only. ‘Negative’ ratings were 1 or 2 stars. allMSU – allMSU uses a 0-4.0 rating system in half-point increments. Here, a ‘positive’ review was a 3.0 or above, and a ‘negative’ rating was 1.0 or below. Metacritic – metaCritic allows users to rate media on a scale of 0-10 in 1-point increments. A ‘positive’ rating was a 7 or above, and a ‘negative’ a 3 or below.

The following chart and table display the distribution of reviews on each of the three sites.

aswenowthink.com

3

Distribution of Reviews 70%

Yelp

60%

allMSU

Metacritic

50% 40% 30% Yelp

20%

allMSU

Metacritic

allMSU

Metacritic

Yelp

10% 0% Positive Reviews

Site Yelp allMSU Metacritic

Neutral Reviews

Positive Reviews 63.6% 53.6% 55.7%

Neutral Reviews 19.1% 23.2% 19.9%

Negative Reviews

Negative Reviews 17.3% 23.2% 24.4%

As the chart and table illustrate, there is a definite tendency towards positive reviews on each of these sites. Unsurprisingly, users are not particularly motivated to share neutral reviews about services, professors, or films. Perhaps more surprisingly, however, negative reviews weren’t very common, either – suggesting that social ratings websites are by-and-large magnets for fans of goods and services. That the distributions were so similar across a diverse range of ratings websites is extremely important. This fact suggests that positivity is not limited to one site or even one vertical. Instead, this appears to be an inherent bias in user behavior.

Conclusion Business owners, researchers, site operators, and consumers need to be aware of the tendency towards positivity in social reviews websites. For business owners and marketing professionals, a few conclusions can be drawn: !

Social ratings websites present an opportunity: Since users are unlikely to write negative reviews, such sites should be embraced as a platform for passionate customers to discuss your goods and services

aswenowthink.com

4 !

!

Encouraging customer engagement is essential: Because many social ratings sites give more prominent placement to goods and services that receive the most reviews, the most talked-about business are likely to gain the most. Messages promoting reviews on menus, signage or in other prominent spots can encourage reviews and benefit the business Take the reviews with a grain of salt: Business owners should not attempt to monitor customer satisfaction using social ratings sites. Because such a clear majority of reviews are favorable, doing so will more often than not lead to false conclusions about the opinion of the general consumer.

Methodology The data used in this report were collected between July 27th and August 3rd, 2009. For allMSU and Metacritic, I generated random samples of all possible reviews. In total, 3,038 and 3,064 reviews were analyzed, respectively. A complete list of reviews was not available on Yelp, so I focused specifically on Chicago, IL. From Chicago, I randomly selected a list of ten neighborhoods and compiled data from three randomlyselected categories. In total, 13,301 reviews were analyzed.

References 1

Greg K.’s reviews can be found at: http://www.yelp.com/user_details?userid=H16zkcIBtvysqqL3UzYsKw 2 See: Here Comes Everybody: The Power of Organizing Without Organizations. New York: Penguin Press. 2009 3 See: What Would Google Do? New York: Harper Collins. 2009. 4 “How Damaging Are Customer Reviews?” from Forrester Research can be purchased at: http://www.forrester.com/Research/Document/Excerpt/0,7211,40649,00.html

aswenowthink.com