A foolproof way to improve games journalism: ban the number 7
It has become meaningless.
By Helen Lewis Published 17 July 2012 10:41
What score do you give a game when you like it, and you think it's interesting and important, but you have to admit to yourself that it just isn't very good? It's a seven out of ten, isn't it? And that sucks.
I was at BAFTA event last night about games journalism, alongside Naomi Alderman, Kieron Gillen, Rob Fahey, Keza McDonald and Keith Stuart, and we kicked off by talking about Metacritic, the site which collects reviews - and delivers an average score based on them.
McDonald wrote a great piece asking whether Metacritic is "ruining the games industry" (albeit inadvertently). Games with higher Metacritic scores sell better, and sometimes a development team will only receive a bonus if they get over, say, 85.
Here's the key paragraph:
A Metacritic average undermines the whole concept of what a review is supposed to be: an experienced critic’s informed and entertaining opinion. Instead it turns reviews into a crowd-sourced number, an average. You can’t average out opinions. If you adore the new Muse album and your Radiohead-loving friend hates it, that doesn’t make it an average album. And yet this is exactly how Metacritic scores are treated by publishers. It punishes divisive games – and honestly, most interesting things are at least a bit divisive.
You can easily apply this to some of the most interesting games of the last few years. Take LA Noire, which made huge strides forward in several really exciting ways (story, motion capture, aesthetics, music), but yet somehow contrived to be less than the sum of its parts. I wrote a short-ish review of it, noting exactly that, then breathed a sigh of pure relief that I didn't have to give it a rating out of ten. (The New Statesman doesn't do 'em, being all highbrow and that.)
But look at Metacritic. LA Noire scores 83 (in a range from 92 to 60), and I honestly can't say what I think that means. Halo 3: ODST gets that score, too, and that game was a perfectly pleasant (but sliiiiiightly unoriginal) iteration of an established series. Heavy Rain is another example where the Metacritic score seems oddly meaningless: its average of 87 comes from a range that goes between GameCritic's 40 and BoomTown's 100, both of which I rather uncharitably suspect were trolling for hits on a popular game by awarding an outlier mark. (A similar thing sometimes happens when writers are filing to a newspaper: they know that only a 0-star or 5-star review will make the front page; they know that a real hatchet job will get the hits rolling in. It's hard to resist that kind of pressure.)
The NS's film critic, Ryan Gilbey, is one of many who hates the five-star system on movie reviews - and who says that trying to choose between 92% and 93% would make him go nuts. Here's what he had to say about numerical ratings when I emailed him:
Scoring, whether you're awarding stars or the somehow-even-more-irritating and pompous grades (C+, A-, etc) which are becoming prevalent online, does such a disservice to the complexities and amorphousness of film - of any art form, in fact. Its sole function is as a consumer guide, a short-cut for readers too lazy to discern for themselves what a writer thinks about the subject in hand.
While allowing exemption for those writers who are forced by their paymasters to award ratings (we've all been there), it seems baffling to me that people who write their own blogs or put grades and ratings in their tweets are doing so out of choice - don't they realise it demeans them and the film? It makes it appear that they don't place any value on a film other than a commercial one. It also encourages the reader and writer to play into the whole star-rating pantomime - the reader inevitably relishes the 1-star review, and the writer performs accordingly, while the 3-star review is given only a cursory read. It's a way of pureeing the review in advance for the reader, warning him or her whether the water will be hot, cold or lukewarm.
I don't want to be read by anyone who needs their hand held through a review, or thinks films can be broken so easily into good, bad and ugly. Ratings are full of anomalies and shortcomings anyway. If Citizen Kane and Andrei Rublev and Some Like It Hot are 5-star movies, how can anything else even be worthy of 3 or 4 stars, let alone 5? Maybe we should invent a sixth star for the canon - and therein lies the insanity of Spinal Tap and the dial that goes up to 11.
The star rating system has commercial currency, no doubt - why else would distributors plaster their posters with quote-less stars ("***** - Daily Mail") which make every advertising hoarding resemble a clear summer night? But challenging it is down to the people who profess to care about the art form they're writing about.
Of course, there are many people willing to defend the score system as a service to the hard-pressed reader: if you only have limited time, then what greater shorthand can there be than a number? At one point in the Bafta debate, McDonald pointed out that IGN's readers really like scores - and Rob Fahey reminded her that it was more that the vocal ones do. (Only around 1 per cent of IGN readers are commenters, so it's impossible to guess the feelings of the community at large.)
My hope, however, is that the mania for scores is just because we're so used to them - and, actually, we wouldn't really miss them if they went. In the short-term, one single step would make games reviews more interesting: BAN THE NUMBER SEVEN (or anything in the 70s, if you're reviewing out of 100).
Most of the panel agreed last night that 7 is shorthand for "this was interesting but had flaws", and therefore was even more meaningless than other arbitrary numbers. There's also some weird voodoo where, even if a score is nominally out of 10, people still regard 5 as a "bad" mark rather than the average. Seven is thus the new 5.
Oh, and while we're at it, maybe ban 0 and 10 as well, tarty little attention-seekers that they are.
Once we've got used to that, maybe then we can start on 1, 2, 3, 4, 5, 6, 8 and 9.
Latest tweets
More from New Statesman
- Online writers:
- Steven Baxter
- Rowenna Davis
- David Allen Green
- Mehdi Hasan
- Nelson Jones
- Gavin Kelly
- Helen Lewis
- Laurie Penny
- The V Spot
- Alex Hern
- Martha Gill
- Alan White
- Samira Shackle
- Alex Andreou
- Nicky Woolf in America
- Bim Adewunmi
- Glosswitch
- Kate Mossman on pop
- Ryan Gilbey on Film
- Martin Robbins
- Rafael Behr
- Eleanor Margolis
- Tools and services:
- Polls
- Predictions
- Archive
- Magazine
- PDF edition
- RSS feeds
- Advertising
- Subscribe
- Special supplements
- Stockists




















17 comments
The aggregate score simply sums up the number of good reviews versus bad reviews. Youre reading too much into it. Say i want to purchase a game i think would be interesting but it has a low aggregate score. This tells me the game was not fun for the majority of the people who played it and, therefore, will probably not be fun for me. It may be, but spending $60 on a game that got a majority of bad reviews is not smart. Conversly, you would feel more comforable if the game you are interested in got a majority of good reviews. It is all about trying to make an informed decision.
I completely agree with the article, but surely the writer should lead by example. Extensively quoting your co-worker does not equal good games journalism.
Even though your article takes up only a tiny piece of internet, it's still a waste.
I don't know why people rage so much about games and how women are depicted in games. First of all, gamers are (most of them) smart people and realize that the real woman is not like that and even starting to make a comparison would skew the reality.
"I don't want to be read by anyone who needs their hand held through a review, or thinks films can be broken so easily into good, bad and ugly."
Lucky for us, he doesn't review games. One of the key differences between game and film reviews is consensus amongst critics. What defines a game as good or bad (presentation, mechanics, 'replayability', fun factor) is a bit more defined when it comes to games. I understand what's fun for some may not be for others, but most opinions about games generally fall in the same area.
And as far as scores go, sometimes I don't care enough about a game to read a full review, but I would still like to know what the reviewer thought about it. In that case, a score with a few bulletpoints is exactly what I need. I'm an advocate for individual (as opposed to aggregate) review scores as long as they accompanied by a "why".
What about adding one simple, automatic tool to these 'crowd-sourcing opinion' sites like Metacritic: standard deviation, the level of variation from the average. A high score would mean people had a diverse range of opinions about it, a low score that most people agree. Jazz it up and call it the Divided Opinion Meter, or the How Likely Are You To Get Into A Barfight Over It? score. Wouldn't require sourcing any more data. Alternatively, just give us a pretty graph with dots, so we can see the clumps. Like fireflies in the darkness of popular opinion: just as numerous, and just as short-lived.
What about the one life left radio show, they literally rate ALL games 7 out of 10. That is a much nicer use of 7.
Surely the fact that the panel "agreed last night that 7 is shorthand for 'this was interesting but had flaws'" is advocacy for the scoring system? If that's how people generally interpret a score of 7, perhaps that is a useful gauge of what they can expect. It also allows them to make comparisons with other media they have experienced to see how it matches up. There's nothing wrong with the number in itself, it's just that we've come to expect excellence - particularly in the gaming arena where you can shell out up to £50 for the privilege of playing the latest hot property. Perhaps 7 is the new 5 as we have become used to seeking high-quality output.
Some sites combine a score with an outline of the positives and negatives. Surely this is a means by which the (perhaps time-p0or) reader can assess whether the item being reviewed is for them and if they should read on. The sheer number of reviews out there regarding any branch of the media makes it prohibitive for consumers to read every one in its entirety. I'd suggest there's nothing wrong with a little shorthand and might even gently subit that it's your responsibility to cater for that.
I think there are a place for numbers in game reviews, simply because games have many different technical aspects which can be objectively judged (frame-rates, loading times, etc.) . When it comes to the more nebulous aspects, scores are a waste of time and you get a much better idea of what a game is like from the text.
Although it could still be abused in much the same manner, a slight improvement on the numbered rating system used by Metacritic would be sentiment analysis of the various reviews put up about a game.
Often when a game is reviewed there are aspects of it that the reviewer will judge to be positive (5 star) and aspects of it that will be deemed poor (1 star). The average will then be calculated and some rating arrived at that doesn't really tell you much about the game or how you're likely to feel when playing it.
Ideally an algorithm would scan reviews and pick out core aspects such as graphics, storyline, combat, longevity, voice acting, etc..., and rate these as either mostly negative or mostly positive on an individual basis and only where they apply.
Of course another way to get an idea of whether you're likely to enjoy a game or not is to pick a handful of games journalists who share your particular tastes in game genres and read their reviews.
I still prefer to play the demos.
Agreed, and thank you very much for last night!
People will talk!
Thoroughly agree. Ratings do nothing but undermine the written / spoken review. Read / listen to the words and make up your own damn mind. And yes, LA Noire is a superb example; I love it, but I can see how many would hate that game. To me, it's far more worthwhile than the Halo series. But that's just me.
Down with scores, up with words. Five star article.
Well, why not go even further than that? Most of the lowest numbers don't mean anything either so get rid of them too. Basically what you want is two ways to say something is good (one with caveats, one mostly without), and two ways to say something is bad (one with caveats, one mostly without) ... so a 4 star system I guess.
Or just scrap review scores entirely and everyone wins.
This is exactly what I was going to say. In response to "Do I like something?" the answer could always be one of:
Yes
Yes, but..
No, but...
No
The review content expands on the answer but essentially they could be represented by 4 stars down to 1. I see no problem with this. We are time poor and aggregation across a large number of samples does provide a reasonable picture of the general critical opinion.
I would like to see the truly divisive stuff made more visible on sites like Metacritic though. The films, music etc that gets an average score because it's polarized opinion. That's the really interesting stuff that it's worth investigating to find out your own opinion and might get overlooked if you go on aggregated score alone.
The idea of flagging up the divisive stuff is a good one - Reddit do this with their "most controversial". It's always interesting to see what provokes a strong, but varied, reaction.
This may be the stupidest thing I've ever read in the New Statesman. Have you all gone insane?