The five star rating system is broken
How have we ended up in this Black Mirror / Meow Meow Beenz dystopian world where the 5 star rating system has infiltrated so much of our daily lives, distorting and controlling our reality? Rating systems seem to have taken on different meanings on different platforms, but all of them seem to revolve around the same fundamental issue, that anything below 4 stars is “bad”. Apparently the tides have changed in our collective minds and there’s likely no going back to a world where 3 stars is seen as good, baseline does the job, happy days. There’s a particular instance that sticks in my mind to support my point of view that our brains have turned to 5 star mush.
The inspiration behing this blog post
A couple years ago a former flatmate wondered why (actually was perhaps somewhat shocked) I had rated a book that I had just finished reading, 3 stars on Goodreads; when I said I had enjoyed it. To my, at the time, flatmate, surely enjoying it warranted a 5 star review… No it does not and I shall explain to you now, as I did then to my flatmate, how my rating system for Goodreads works (and how I think it should work for you too).
⭐️⭐️⭐️⭐️⭐️ - I wish this book never ended, I would absolutely read it again, I would gift it to someone and I would go out of my way to recommend to others (even if they didn’t ask or it naturally came up in conversation). This book will stay on my bookshelf for life no matter where in the world I go.
⭐️⭐️⭐️⭐️ - This book was really damn good, I’m really glad I read it, but probably wouldn’t read it again. I would recommend it if it came up naturally in conversation or if I was asked about it. It will stay on my bookshelf (but maybe not in the best spot) and may be cut if I need to make space.
⭐️⭐️⭐️ - This book was a good and pleasant read, I wouldn't read it again and I would say I’ve read it if asked, but I wouldn’t go either way on recommending it or not to someone. This might stay in my collection, but will likely end up being resold on eBay, if not then somewhere non prominent in my home.
⭐️⭐️ - This book was not good and not an enjoyable read, I would not read it again and I would actively not recommend someone to read it if they asked me about it. This book will not stay in my collection, it will not sit on my bookshelf, it will be resold to someone who will appreciate it.
⭐️ - This book should never have existed, it was a total waste of my life and I want a refund on both my money and time from the author. I will actively tell people not to read it and how bad it is (whilst being passionately frustrated about all the ways it was a waste of paper). The only way this book will stay on my bookshelf is if it’s in a paper shredder (having been shredded) that is sitting on my bookshelf… It may also be used as kindling for a fire later on.
As a note on Goodreads and their rating system, it is rather frustrating that one cannot give half star marks (or even better, quarter star) for books (especially for 3.5 and 4.5 as I feel many books I read belong in this edge case zone). Although it’s worth noting that Goodreads is owned by Amazon and so it should come as little surprise that my rant will continue with them shortly.
How the 5 star rating Systems currently work
Due to this nonsense system that has been further perpetuated by our need for instant gratification combined with corporate needs for social proof we’re now in a situation where we measure the quality of something by the fractional changes between 4 and 5 stars. We have a 10 point rating system that exists between the 4th and 5th star.
Don’t believe me? Go look at your favourite restaurants on Google maps. Look at their ratings and notice that they’re likely all 4 stars; now pay further attention to the number that comes after the 4. I feel fairly confident in saying that those 0.1 increments tell you a lot about how you view the value of something and how likely you are to spend your money on it. We essentially live with a 10 star rating system within a 5 star rating system, am I the only person who thinks this is insane?
Why is it broken? Blame the platforms
Amazon is a particularly frustrating platform for this as the rating system has essentially become meaningless as products are either rated based on brigaded reviews to push a product’s sales up; or based almost entirely on the shipping time and quality of the packaging; or rated as soon as they’re received and often before they’ve even been used. I suppose we can lump Google business reviews into this too as they’re just as guilty of this nonsense… Uber too; Trust Pilot, you’re the worst of them all, get in the sack with the rest of them! When everyone is 5 stars, no one is 5 stars.
The platforms are to blame for this sloppy user feedback (despite my anger at the individuals who rate a product based on shipping rather than the product itself - THANK YOU FOR THE USELESS INFORMATION ABOUT THIS PRODUCT BUT GOOD TO KNOW THAT THE SHIPPING FROM A MULTI-BILLION DOLLAR CORPORATION WAS A-OKAY!).
Why are the platforms to blame? Well, simply because they do not teach people how to rate things properly, or benchmark what a rating should mean, or segment out different aspects of an experience to provide accurate and useful feedback. All they really want is to make more sales, so why would they change a system that benefits them? Well, because in the long run (as is happening now) your rating system becomes meaningless and then no-one trusts it.
Fixing Amazon ratings (as an example)
It’s not that hard Amazon! Just ask the customer a few simple things to help refine the value of the feedback a customer gives. You can then use this to weigh the value of a review rather than having minimum effort feedback and maximum effort feedback be weighted the same. Here are a couple of suggestions to add to your product rating feedback from customers and the way you could display it to potential customers.
1. How was the shipping time and packaging of your product?
This will allow you to segment out information about your shipping from specific distribution centres. Super useful data to get and why you’re not doing this already is beyond me. Not to mention that the vast majority of shipping is already done by amazon fulfilment centres, so you could also automatically filter sales that were shipped by a third party as part of this customer feedback to help with customer experience.
2. How long have you been using the product you purchased for? Have you even used the product yet?
I get that it’s easier to get someone to review a product when the product is fresh in their head. We want to give feedback on something as quickly as possible, striking when the iron is hot and gaining that product value momentum for sales. But it would be really useful if the review stated how long they’d been using the product for (or even when their linked account ordered it). These reviews of “got it in 3 days” are so frustrating, have you even used the product? How long have you used it for? I’m not going to put as much weight on the person who’s written a review after 1 day of use (or even before they receive it) than someone who’s been using it for a week or longer (let alone someone who’s giving a rating without even having used the damn thing).
3. Who fulfilled the product / who is the seller?
This one I think is pretty self explanatory, if amazon didn’t fulfil the product and it didn’t come from their inventory or the same seller that the product is listed under, then it should not be possible to publish a rating to that listing. The review can be “published” but perhaps shadow banned.
As a point to this, I think reviews need to be weighted to verified reviewers. Segment the ratings from verified and unverified. If they don’t statistically match-up then we know that something fishy is going on.
4. How long ago was the review?
Products often change over time as manufacturers change suppliers or material choices or whatever. It’s important that reviews that were published a long time ago are weighted against newer reviews. It’s possible that a product starts with bad reviews and then moves to better, but it’ll be permanently weighed down by these existing reviews; or vice versa. There needs to be something that compensates for reviews changing over time.
5. Does the rating line up with the context of the feedback
So many reviews use either 1 star or 5 stars and then the context of what they write does not line up to it. Or the other way where someone will write a review where something is perfect but only give it three stars. Our world is being governed by chaotic individuals who don’t understand the connection between their cognition and the star rating system.
The five star rating system needs to Change
I propose that we in general approach life as such for the 5 star rating system (until a better system comes along).
⭐️ - Fuck you, this is shit and never should have existed.
⭐️⭐️ - In general this was less than expected, I’m disappointed but maybe it wasn’t for me.
⭐️⭐️⭐️ - This is a good thing, it does what it’s supposed to do and you should use it also as it is adequate for the job, you will not be disappointed, but you also will not be overwhelmed; you will be… whelmed.
⭐️⭐️⭐️⭐️ - This is a premium version of a thing. It really does something very well and brings me joy.
⭐️⭐️⭐️⭐️⭐️ - This is like a class A drug being injected straight into my veins. Give me this all day all the time.
Finally I would like to add that a 7 star rating system might be more appropriate to move to as it gives us more variables to control and as such measure quality through, although we may end up in the same place eventually. As such, I’m sure that probably the only way to really resolve this is through small step changes and moving towards a system that isn’t so defined, something more abstract that still gives quality information (I’d be happy with being able to give quarter increments on ratings for the time being though). Perhaps I should build a non-star based system for rating things on online platforms.
This blog post brought to you by a five star man /s
