Monday, May 12, 2014

The much maligned and misunderstood 3 star rating.

There is quite a bit of variability in ratings systems on various platforms and this is compounded by the fact that people are averaging the responses on a Likert scale, which statistically is inappropriate. Most of the confusion revolves around what is a positive or negative review and which a 3 star actually is.

For Amazon 3 stars is "It's OK" and they count it as a critical review. On Barnes & Noble 3 stars is "Good". On Goodreads 3 stars means "I Liked it" and 2 stars means "It's OK. Personally, I rate books on how likely I am to re-read it. Books I love I will always re-read, books I don't love I never will. For me 3 stars means "This book is fairly decent. I am likely to read the rest of the series or other books by this author. However, I'm only likely to re-read this book if a new book is coming out for the series." In other words, my opinion is fairly neutral but the product still has value. It is NOT a negative ranking.

I think the biggest problem here is that none of the platforms recognize a neutral rating, a rating that is essential to the use of Likert scales. Basically, they are all doing it wrong. If they don't want a neutral rating, it should only be a 4 star system making it a forced choice scale. The second problem is the assumption that neutrality doesn't have value. It does, being in that "neither agree nor disagree" is precisely where one should be able to enumerate both the positives and the negatives and if they balance out, then that is when you are in the middle.

Think of it more like school grades.

A represents outstanding distinction and excellence. 90-100%
- These are not impossible to achieve but are rare and difficult to come by.
B signifies levels of solid accomplishment and goodness. 80- 89%
- Good is more common than excellent but more rare than average. While there is merit to hard work and long hours, it does not always guarantee success. Goodness refers to the combined results not just the effort.
C signifies "average- simple, common, adequate but ordinary 70-79%
- Average is not usually an appealing rank to artists who strive for extraordinary and unique. C is however a very respectable point. Recognize what more is needed; plan to move ahead, improve and grow.
D represents results less than standard and/or mediocre- just passable 60-69%
- Perhaps priorities about school or life have not been established. Recognize however, that a D can also mean that you truly do not understand what is expected. You should make an office appointment to discuss how you might take action on your future and upcoming assignment problems.
F is a clear failure. < 59%
-It represents lack of effort/interest. It is a cause for deep concern.
http://artandtech.osu.edu/4101/lettergrades.html

A 3 star review is like getting a C, and since when is a passing grade just not enough. Guess what, you have doctors and lawyers and engineers that you rely on every day who were C students, and they get the job done. Hell, we have had C-student presidents. That's what a 3 star book is to me, it got the job done, it entertained me, it neither repulsed nor enthralled me and there is room for improvement.

Personally, I don't tend to trust the 5 star reviews as much as I do reviews that are more critical. When I am reading reviews on Amazon, I tend to first look at the most helpful critical review and then I look at all the 3 star and below reviews.

The unfortunate side effect of honestly rating books on Amazon is that when people disagree with your rating or review, they will mark your review "unhelpful" no matter how well written or helpful it actually may be. Amazon might as well go ahead and call those buttons "like" and "dislike". So I guess I'll never be an Amazon Top Rated Reviewer, but that's ok, at least I'll know I am honest. I suppose prior to this I have been a bit naive and unobservant when it comes to the Amazon review system, I'll have to make more of an effort to actually mark reviews helpful and unhelpful, and I'll have to pay no attention to my scores because it's awfully tempting to retaliate.

No comments:

Post a Comment