Numbers can't be treated like words, and we need to stop thinking they can. Photo: Getty
Show Hide image

On Nate Silver’s FiveThirtyEight: is it possible for a data-driven journalist to tell a good story?

The launch of Nate Silver's new FiveThirtyEight site for data-driven reporting has attracted a lot of criticism from big name journalists and pundits. Should numbers rule reporting?

First, a disclaimer: this piece isn’t grounded in any data analysis whatsoever. Sure, I’ve done my research, but this is mostly about the unquantifiable – the way many of us seem to feel.

When Nate Silver relaunched an expanded version of his “data-driven” reporting website FiveThirtyEight last week, he did so with a bit of a bang – a “manifesto”, to be precise, one that was likely drafted (per his suggestion) in the middle of the night with the aid of a lot of Red Bull. It’s cogent enough, if a bit rambling: Silver lays out the ills of the state of modern journalism, with its hunches and false trends and big, blustery ideas, stuff that’s rarely backed up by hard facts. He posits his own analytical methods – the ones that predicted the winner of the 2012 US presidential election correctly in every single state – as the antidote:

Narrative accounts of individual news events can be informative and pleasurable to read, and they can have a lot of intrinsic value whether or not they reveal some larger truth. But it can be extraordinarily hard to make generalisations about news events unless you stop to classify their most essential details according to some numbering or ordering system, turning anecdote into data.

Silver has to rely on the dreaded anecdote to illustrate the pitfalls of baseless punditry, notably Peggy Noonan’s assertion that “all the vibrations” and all the lawn signs in her neighborhood were pointing to a Romney victory in 2012. In an interview with New York Magazine on the eve of the site’s launch, Silver had a go at op-ed sections more broadly, especially the New York Times, his former employer. (“They don’t permit a lot of complexity in their thinking. They pull threads together from very weak evidence and draw grand conclusions based on them.”)

If our choice is Silver or the New York Times op-ed pages, well, that doesn’t even seem like a fair fight. So many of us are weary of opinions dressed up as reporting, or of the media’s habit of using statistical outliers to, in Silver’s words, erroneously “indicat[e] some broader trend”. The New York Times itself is famous for its “trend pieces”; just this weekend their own public editor poked fun at the paper for a recent style article that suggested that the monocle is on its way back.

We like Nate Silver, and maybe we even need Nate Silver – so why did everyone spend the past week hating on Nate Silver? Well, not everyone: the hundred-plus comments on the manifesto were gushingly positive. (A sample: “Bye bye, rest of the internet, I probably won't miss you.”) But every day last week I was bombarded with articles that pecked away at FiveThirtyEight, scrutinising the methods of collection or the presentation of data in individual pieces, or, more often, knocking at the core premise of his project, of the “intersection of data and journalism”. There are only a handful of articles to critique, but people are declaring FiveThirtyEight dead on arrival. Paul Krugman weighed in twice! First to take issue with an individual argument, and later to declare: “So far it looks like something between a disappointment and a disaster.”

The offerings of the first week were admittedly not fantastic – I wasn’t personally impressed with much, and was actually a bit angered by the one that hugged closest to my current academic discipline (the digital humanities), a textual analysis of Shakespeare’s plays. (Because while there are plenty of interesting ways to use literary text analysis, this is not one of them – the data she looks at is fine, but the way she connects this information to either the plays themselves or their historical context is essentially useless.) But one would think that a new venture would be afforded a little room to find its bearings. The good will that Silver earned with his rock-solid political predictions seems to have slipped away. From the big name journalists, the pundits on the defensive, there has to be at least a little Schadenfreude here – or rather, the niggling desire to see this project fail, and delight when one lame data-driven piece after another is published.

But then, Silver didn’t just malign the pundits: he suggested most, if not all, journalists were lacking in quantitative analytical skills: “Students who enter college with the intent to major in journalism or communications have above-average test scores in reading and writing, but below-average scores in mathematics.” (For my own self-preservation, let me go on the record and say that my maths score was – to everyone’s surprise – ten points higher than the verbal. Ten less than the writing, though.) Setting aside the fact that most people I know working in the media came from all sorts of disciplines, there’s certainly truth here. I have to veer into the realm of the anecdotal to say that many writers I know seem to lack a fondness for numbers, and some even harbor a bit of distrust for technology. And I think a good majority of them would place themselves in Silver’s exception-to-the-rule camp, that their work is “rigorous without being quantitative”, “Robert Caro to Richard Ben Cramer.”

(Or, say, Janet Malcolm, or some other “rigorous” female journalist. Let’s not forget that there was another major thread of dissent running through the launch of FiveThirtyEight: Emily Bell’s assertion that Silver, along with new ventures from Ezra Klein and Glenn Greenwald, are failing to be as revolutionary they claim: “It’s impossible not to notice that in the Bitcoin rush to revolutionise journalism, the protagonists are almost exclusively – and increasingly – male and white.” Silver responded that, “about 85 per cent of our applications come in from men. That worries us,” which suggests that while he’s complicit, he’s not responsible. But perhaps the gender and race issues here are too thorny for the space allotted.)

Is it possible for a data-driven journalist to tell a good story? If you’re calling an election or breaking down a sporting match, good, rigorous data analysis can shine. But tackling the rest of the news? Or for that matter, culture? Traditional journalists might be uniformly bad at maths, but does maths matter when crafting a narrative? In the backlash against FiveThirtyEight, a few themes have emerged. There’s the idea that all the data analysis in the world can’t always tell a good story – sometimes the numbers don’t offer any space for strong conclusions. There’s the idea that data can never tell the whole story – that the reader needs more to grab onto than the aggregate. Silver speaks of the difficulty in telling stories from “generalisations” that aren’t steeped in data; I’d argue it’s an enormous challenge to tell any story with only generalisations.

And then there’s the idea, tapping into traditional journalists’ indignation, that there isn’t much space in the art of the written word for number-crunching. It’s hard to take this at face value, when you see numbers baldly manipulated or simply misunderstood in news articles every day. But it’s harder to swallow Silver’s antidote when journalism-by-the-numbers is so often such an unsatisfying read. The prose regularly (and occasionally smugly) sits back on the phrase “we looked at the numbers” or throws down a chart or a graph, as though the act of providing these numbers speaks for itself. I see this a lot here in our relatively new field of the digital humanities, too: research in history and English and other fields somehow quantified, but nothing substantive beyond the data, just people sitting proudly on the fact that they crunched any numbers at all. I saw a tweet a few weeks back that drove me crazy, something along the lines of, “Metadata is the metaphor of our times: more than the bare facts, but less than the full story.” This is fundamental misunderstanding of metadata, but if we substitute data analysis, it might ring a bit truer.

Perhaps there needs to be more collaboration between traditional journalists and statisticians, rather than each team going it alone or a push for the next crop of journalists to try to master both skills. Right now, there’s just something lacking in the questions a lot of data-driven journalists are asking – and, for now, at least, there’s still something lacking in the way they present the answers. I’m reminded of Choire Sicha’s fantastic line to the tech bros whose start-up manifestos clogged Medium in its early months: “Words are like code. You have to put them in the right order for things to work.” Numbers can’t be treated like words, but there’s clearly a need for both of them in journalism. We just need to find a better way for them to live side by side.


Elizabeth Minkel is a staff writer for The Millions, and writes a regular column on fan culture for the New Statesman. She is on Twitter @ElizabethMinkel.

Photo: Getty
Show Hide image

The Prevent strategy needs a rethink, not a rebrand

A bad policy by any other name is still a bad policy.

Yesterday the Home Affairs Select Committee published its report on radicalization in the UK. While the focus of the coverage has been on its claim that social media companies like Facebook, Twitter and YouTube are “consciously failing” to combat the promotion of terrorism and extremism, it also reported on Prevent. The report rightly engages with criticism of Prevent, acknowledging how it has affected the Muslim community and calling for it to become more transparent:

“The concerns about Prevent amongst the communities most affected by it must be addressed. Otherwise it will continue to be viewed with suspicion by many, and by some as “toxic”… The government must be more transparent about what it is doing on the Prevent strategy, including by publicising its engagement activities, and providing updates on outcomes, through an easily accessible online portal.”

While this acknowledgement is good news, it is hard to see how real change will occur. As I have written previously, as Prevent has become more entrenched in British society, it has also become more secretive. For example, in August 2013, I lodged FOI requests to designated Prevent priority areas, asking for the most up-to-date Prevent funding information, including what projects received funding and details of any project engaging specifically with far-right extremism. I lodged almost identical requests between 2008 and 2009, all of which were successful. All but one of the 2013 requests were denied.

This denial is significant. Before the 2011 review, the Prevent strategy distributed money to help local authorities fight violent extremism and in doing so identified priority areas based solely on demographics. Any local authority with a Muslim population of at least five per cent was automatically given Prevent funding. The 2011 review pledged to end this. It further promised to expand Prevent to include far-right extremism and stop its use in community cohesion projects. Through these FOI requests I was trying to find out whether or not the 2011 pledges had been met. But with the blanket denial of information, I was left in the dark.

It is telling that the report’s concerns with Prevent are not new and have in fact been highlighted in several reports by the same Home Affairs Select Committee, as well as numerous reports by NGOs. But nothing has changed. In fact, the only change proposed by the report is to give Prevent a new name: Engage. But the problem was never the name. Prevent relies on the premise that terrorism and extremism are inherently connected with Islam, and until this is changed, it will continue to be at best counter-productive, and at worst, deeply discriminatory.

In his evidence to the committee, David Anderson, the independent ombudsman of terrorism legislation, has called for an independent review of the Prevent strategy. This would be a start. However, more is required. What is needed is a radical new approach to counter-terrorism and counter-extremism, one that targets all forms of extremism and that does not stigmatise or stereotype those affected.

Such an approach has been pioneered in the Danish town of Aarhus. Faced with increased numbers of youngsters leaving Aarhus for Syria, police officers made it clear that those who had travelled to Syria were welcome to come home, where they would receive help with going back to school, finding a place to live and whatever else was necessary for them to find their way back to Danish society.  Known as the ‘Aarhus model’, this approach focuses on inclusion, mentorship and non-criminalisation. It is the opposite of Prevent, which has from its very start framed British Muslims as a particularly deviant suspect community.

We need to change the narrative of counter-terrorism in the UK, but a narrative is not changed by a new title. Just as a rose by any other name would smell as sweet, a bad policy by any other name is still a bad policy. While the Home Affairs Select Committee concern about Prevent is welcomed, real action is needed. This will involve actually engaging with the Muslim community, listening to their concerns and not dismissing them as misunderstandings. It will require serious investigation of the damages caused by new Prevent statutory duty, something which the report does acknowledge as a concern.  Finally, real action on Prevent in particular, but extremism in general, will require developing a wide-ranging counter-extremism strategy that directly engages with far-right extremism. This has been notably absent from today’s report, even though far-right extremism is on the rise. After all, far-right extremists make up half of all counter-radicalization referrals in Yorkshire, and 30 per cent of the caseload in the east Midlands.

It will also require changing the way we think about those who are radicalized. The Aarhus model proves that such a change is possible. Radicalization is indeed a real problem, one imagines it will be even more so considering the country’s flagship counter-radicalization strategy remains problematic and ineffective. In the end, Prevent may be renamed a thousand times, but unless real effort is put in actually changing the strategy, it will remain toxic. 

Dr Maria Norris works at London School of Economics and Political Science. She tweets as @MariaWNorris.