Numbers can't be treated like words, and we need to stop thinking they can. Photo: Getty
Show Hide image

On Nate Silver’s FiveThirtyEight: is it possible for a data-driven journalist to tell a good story?

The launch of Nate Silver's new FiveThirtyEight site for data-driven reporting has attracted a lot of criticism from big name journalists and pundits. Should numbers rule reporting?

First, a disclaimer: this piece isn’t grounded in any data analysis whatsoever. Sure, I’ve done my research, but this is mostly about the unquantifiable – the way many of us seem to feel.

When Nate Silver relaunched an expanded version of his “data-driven” reporting website FiveThirtyEight last week, he did so with a bit of a bang – a “manifesto”, to be precise, one that was likely drafted (per his suggestion) in the middle of the night with the aid of a lot of Red Bull. It’s cogent enough, if a bit rambling: Silver lays out the ills of the state of modern journalism, with its hunches and false trends and big, blustery ideas, stuff that’s rarely backed up by hard facts. He posits his own analytical methods – the ones that predicted the winner of the 2012 US presidential election correctly in every single state – as the antidote:

Narrative accounts of individual news events can be informative and pleasurable to read, and they can have a lot of intrinsic value whether or not they reveal some larger truth. But it can be extraordinarily hard to make generalisations about news events unless you stop to classify their most essential details according to some numbering or ordering system, turning anecdote into data.

Silver has to rely on the dreaded anecdote to illustrate the pitfalls of baseless punditry, notably Peggy Noonan’s assertion that “all the vibrations” and all the lawn signs in her neighborhood were pointing to a Romney victory in 2012. In an interview with New York Magazine on the eve of the site’s launch, Silver had a go at op-ed sections more broadly, especially the New York Times, his former employer. (“They don’t permit a lot of complexity in their thinking. They pull threads together from very weak evidence and draw grand conclusions based on them.”)

If our choice is Silver or the New York Times op-ed pages, well, that doesn’t even seem like a fair fight. So many of us are weary of opinions dressed up as reporting, or of the media’s habit of using statistical outliers to, in Silver’s words, erroneously “indicat[e] some broader trend”. The New York Times itself is famous for its “trend pieces”; just this weekend their own public editor poked fun at the paper for a recent style article that suggested that the monocle is on its way back.

We like Nate Silver, and maybe we even need Nate Silver – so why did everyone spend the past week hating on Nate Silver? Well, not everyone: the hundred-plus comments on the manifesto were gushingly positive. (A sample: “Bye bye, rest of the internet, I probably won't miss you.”) But every day last week I was bombarded with articles that pecked away at FiveThirtyEight, scrutinising the methods of collection or the presentation of data in individual pieces, or, more often, knocking at the core premise of his project, of the “intersection of data and journalism”. There are only a handful of articles to critique, but people are declaring FiveThirtyEight dead on arrival. Paul Krugman weighed in twice! First to take issue with an individual argument, and later to declare: “So far it looks like something between a disappointment and a disaster.”

The offerings of the first week were admittedly not fantastic – I wasn’t personally impressed with much, and was actually a bit angered by the one that hugged closest to my current academic discipline (the digital humanities), a textual analysis of Shakespeare’s plays. (Because while there are plenty of interesting ways to use literary text analysis, this is not one of them – the data she looks at is fine, but the way she connects this information to either the plays themselves or their historical context is essentially useless.) But one would think that a new venture would be afforded a little room to find its bearings. The good will that Silver earned with his rock-solid political predictions seems to have slipped away. From the big name journalists, the pundits on the defensive, there has to be at least a little Schadenfreude here – or rather, the niggling desire to see this project fail, and delight when one lame data-driven piece after another is published.

But then, Silver didn’t just malign the pundits: he suggested most, if not all, journalists were lacking in quantitative analytical skills: “Students who enter college with the intent to major in journalism or communications have above-average test scores in reading and writing, but below-average scores in mathematics.” (For my own self-preservation, let me go on the record and say that my maths score was – to everyone’s surprise – ten points higher than the verbal. Ten less than the writing, though.) Setting aside the fact that most people I know working in the media came from all sorts of disciplines, there’s certainly truth here. I have to veer into the realm of the anecdotal to say that many writers I know seem to lack a fondness for numbers, and some even harbor a bit of distrust for technology. And I think a good majority of them would place themselves in Silver’s exception-to-the-rule camp, that their work is “rigorous without being quantitative”, “Robert Caro to Richard Ben Cramer.”

(Or, say, Janet Malcolm, or some other “rigorous” female journalist. Let’s not forget that there was another major thread of dissent running through the launch of FiveThirtyEight: Emily Bell’s assertion that Silver, along with new ventures from Ezra Klein and Glenn Greenwald, are failing to be as revolutionary they claim: “It’s impossible not to notice that in the Bitcoin rush to revolutionise journalism, the protagonists are almost exclusively – and increasingly – male and white.” Silver responded that, “about 85 per cent of our applications come in from men. That worries us,” which suggests that while he’s complicit, he’s not responsible. But perhaps the gender and race issues here are too thorny for the space allotted.)

Is it possible for a data-driven journalist to tell a good story? If you’re calling an election or breaking down a sporting match, good, rigorous data analysis can shine. But tackling the rest of the news? Or for that matter, culture? Traditional journalists might be uniformly bad at maths, but does maths matter when crafting a narrative? In the backlash against FiveThirtyEight, a few themes have emerged. There’s the idea that all the data analysis in the world can’t always tell a good story – sometimes the numbers don’t offer any space for strong conclusions. There’s the idea that data can never tell the whole story – that the reader needs more to grab onto than the aggregate. Silver speaks of the difficulty in telling stories from “generalisations” that aren’t steeped in data; I’d argue it’s an enormous challenge to tell any story with only generalisations.

And then there’s the idea, tapping into traditional journalists’ indignation, that there isn’t much space in the art of the written word for number-crunching. It’s hard to take this at face value, when you see numbers baldly manipulated or simply misunderstood in news articles every day. But it’s harder to swallow Silver’s antidote when journalism-by-the-numbers is so often such an unsatisfying read. The prose regularly (and occasionally smugly) sits back on the phrase “we looked at the numbers” or throws down a chart or a graph, as though the act of providing these numbers speaks for itself. I see this a lot here in our relatively new field of the digital humanities, too: research in history and English and other fields somehow quantified, but nothing substantive beyond the data, just people sitting proudly on the fact that they crunched any numbers at all. I saw a tweet a few weeks back that drove me crazy, something along the lines of, “Metadata is the metaphor of our times: more than the bare facts, but less than the full story.” This is fundamental misunderstanding of metadata, but if we substitute data analysis, it might ring a bit truer.

Perhaps there needs to be more collaboration between traditional journalists and statisticians, rather than each team going it alone or a push for the next crop of journalists to try to master both skills. Right now, there’s just something lacking in the questions a lot of data-driven journalists are asking – and, for now, at least, there’s still something lacking in the way they present the answers. I’m reminded of Choire Sicha’s fantastic line to the tech bros whose start-up manifestos clogged Medium in its early months: “Words are like code. You have to put them in the right order for things to work.” Numbers can’t be treated like words, but there’s clearly a need for both of them in journalism. We just need to find a better way for them to live side by side.


Elizabeth Minkel is a staff writer for The Millions, and writes a regular column on fan culture for the New Statesman. She is on Twitter @ElizabethMinkel.

Photo: Getty
Show Hide image

On the important issues, Louise Casey all too often has little to say

Far from moving the debate on, this new report on integration adds little to the report I commissioned in 2001. 

For 15 years, “numerous government reports on community cohesion and integration have not been implemented with enough force or consistency” concludes Louise Casey’s review of  integration.  The government’s lukewarm response suggests their effort will be as “diluted and muddled” as all the rest.

There’s a deeper reason why governments shy away from the measures that are needed. The report's wealth of data sets out a stark if sometimes contestable picture of a divided society.  But no amount of data can really bring the lives of our fellow citizens to life. As the Brexit vote underlined, this is now a nation divided by class, geography, education, wealth, opportunity and race. Those divisions colour the way we live our lives, the way we see problems in society, the relations we have with others, and our political choices. The report, like many before it, stops short of setting out that reality. It’s easier to pretend that most of us pretty much agree on most things; but just few people don’t agree and they must be the problem. Predictably, much of the early coverage has focussed on the Muslim community and new migrants. If only it were so easy.

According to Casey “in this country, we take poverty, social exclusion, social justice and social mobility seriously” and we do it “across political divides”. Apparently “creating a fair, just society where everyone can prosper and get on” is a cornerstone of British values. Yet for page after page the report chronicles the serial failure of this benign consensus to tackle educational under-performance, and economic and racial disadvantage. If we all agree, how come we haven't done anything about it?

These problems are not certainly easy to solve, but more lip service is paid to tackling them than effort. The practical material issues documented here need addressing, but punches are pulled when hard answers are needed. Given the dramatic impact of mass migration on cohesion, is integration possible while current rates of immigration persist? Can we find the political will to tackle poverty and disadvantage when those who might benefit from the effort are divided against each other by suspicion, race, geography and values? After all, rather than progressive policies producing a cohesive society, social unity is the precondition for the introduction of progressive policies.

We don't actually actually agree on what our “fundamental values” mean in practice. We can all sign up to democracy and the rule of law, but as soon as those are put into practice – see the court case on Article 50 – we are divided. When judges are popularly seen as “enemies of the people” and a vote in an elected parliament as a threat to democracy, in what sense are law and democracy fundamental?

Casey usefully highlights how treating homeless families equally, irrespective of ethnicity and length of residence can create the perception that minorities are being favoured over long standing residents. Our differing views on what is “just” and how “fairness” are defined can tear us apart. Is it fair to favour the newcomer over the indigenous? Is it just to put length of time on the waiting list above housing need? We often don't even acknowledge the legitimacy of other points of view, let alone try to find common ground.

The continual invocation of Britain and British values lends an air of unreality to the report.  Most people in England include British in their identity, but Englishness and English interests are of growing importance. In a worrying development, some areas of England  may be polarising between a white Englishness and an ethnic minority Britishness. Integration won't happen without a shared national story that combines a unifying national identity with the acceptance that we all have more than one identity that matters to us. Ignoring the reality of complex and multiple identities closes off one essential way forward.

None of this means that the criticism of some reactionary and occasionally dangerous ideas and practices in the Muslim community should be ignored and not confronted. But in a country where the established church opposes homosexual relationships and praise for Vladimir Putin's Russia is now mainstream politics it is hard to believe that all our problems can be reduced to the behaviour of a minority of a minority community.

John Denham was a Labour MP from 1992 to 2015, and a Secretary of State 2007 to 2010. He is Director of the Centre for English Identity and Politics at Winchester University