Oli Scarff/Getty Images
Show Hide image

Harvesting the wisdom of the crowd

The “superforecasters” who show remarkable proficiency in answering big geopolitical questions - despite their lack of knowledge.

Can the wisdom of crowds make more accurate predictions than top intelligence experts? Philip E Tetlock, a University of Pennsylvania political scientist, set up the Good Judgment Project in 2011 in order to find out. Funded by the US government, the programme identified a group of “superforecasters” who, despite possessing no particular expertise nor access to any secret information, have shown remarkable proficiency in answering big geopolitical questions – such as whether Serbia would be granted European Union candidacy by the end of 2011, or whether Bashar al-Assad’s regime in Syria would survive in 2012.

Supers, as they are known, are those in the top 2 per cent of participants. Tetlock’s thesis is that by aggregating the guesses of supers, he can derive a probability that comes close to the truth. Although no one can know everything about a given topic, everyone knows a little about something. Tetlock argues that aggregation amplifies what people get right and cancels out what they get wrong.

It sounds too good to be true: a relatively simple approach that overturns decades of received wisdom, a little like the case studies in Freakonomics or Moneyball, but specific to intelligence. What is astonishing is that the approach has been shown to work, with participants achieving accuracy scores well above random chance and supposedly even beating CIA analysts (though the agency apparently refuses to confirm this).

In October, the supers held a public conference at the London School of Economics. Before it opened, I met Andreas, an Italian super who lives in London and works in finance. When I asked him which forecast he was most proud of, he told me that he had closely predicted the number of banks that would fail the EU “stress test” in 2014. But was this forecast his “best”? Rather than giving a quick answer, Andreas instinctively began to deconstruct my rephrased question, asking rhetorically what might be meant by “best”. According to Tetlock, the ability to break down questions is a key skill in becoming a super.

The methods used to collate judgements have varied since 2011 but one of the most successful has been to use a market, or “forecasting tournament”, which works much like a real stock market. Each super is given 50,000 units of a fictional currency known as “Inklings” to invest in various outcomes. If, say, you had bet on Assad surviving a year ago and the odds were then at 10 per cent, you would have won a big return.

What was clear as I sat in the conference room – listening to the likes of the former Ministry of Defence analyst Nick Hare speaking about forecasting defence issues, or the conference organiser Michael Story bemoaning the predictions of media pundits – was that I was surrounded by very smart people. Pablo, the super next to me, was the dean of MBA students at a major business school. One questioner identified himself as working for the Bank of England. Contrary to the promotional pitch of Tetlock’s recent book, Superforecasting: the Art and Science of Prediction, these are not, for the most part, ordinary people, but highly qualified, well-connected and well-read individuals.

One participant suggested that the questions the supers performed worst on were those in their own professional field. When it came to breaking down a question and engaging with it objectively, he claimed, he was unable to see the wood for the trees. Chris, a strategy management consultant who joined the programme in its second year, described this quality positively as “conscious incompetence” – in essence, having an awareness of what you do not know and being open to having your forecasts challenged by others so that together you can arrive at better judgements.

Wanda, a Spanish-speaking guest from Texas (something of a rarity at the conference: the demography of supers seems to be overwhelmingly white and male), told me that she had performed best on questions relating to Iran. To do so, however, she had to be dispassionate, something she found difficult because she has strong opinions about Barack Obama’s recent nuclear deal with Iran. Ironically, for a programme that focuses on contentious political issues, the one taboo among supers seems to be personal politics. When I asked another participant if he would discuss his voting patterns with his super colleagues, he said that to do so would hinder co-operation.

So, what are the chances that this method of intelligence gathering will be taken seriously beyond this select group? I’m not so sure, but I know the right people to ask.

This article first appeared in the 29 October 2015 issue of the New Statesman, Israel: the Third Intifada?

Show Hide image

Marching against climate change in the age of Donald Trump

The People’s Climate Movement is as much about politics as science. That's its strength.

Saying goodbye is never easy. But the present generation are facing an awful lot of farewells: to the melting arctic, the dying Barrier Reef, and the general resilience of ecosystems around the world. As Margaret Atwood described it in her essay of the same name: “It’s not climate change, it’s everything change”.

The problem with “everything-change” is that it can be overwhelming. How do you even decide where to start?

The People’s Climate Movement want to begin by making visible the extent of concern out there. This weekend, a coalition of organisations have planned a protest march on the American capital. Between 50,000 -100,000 people are expected to attend, including eco-celebrities Leonardo Di Caprio, Al Gore and Richard Branson.

In London, a group called Campaign Against Climate Change, are co-ordinating a UK-based solidarity event. Protestors will meet at 11.30am in Old Palace yard opposite Parliament, then move to Westminster Bridge, where they will spell out a message to Theresa May: “Trump and May: Climate Disaster”.

For UK campaigners, this is a vital opportunity to raise awareness of the many ways in which action on climate change is under threat. Claire James from CACC outlines the sense of frustration and injustice that many feel with regard to recent government policy: “There have been 12,000 jobs lost last year in the solar industry alone and installation numbers have plummeted. Meanwhile fracking, hugely unpopular, is given determined backing.”

Ahead of the June election, campaigners are using the event to call for specific, cross-party commitments. One, fast-tracking the UK’s delayed Climate Change Plan. Two, ruling out new trade deals that compromise environmental, worker or consumer rights. And three, implementing a fair deal for UK solar and wind industry. “Our action on Saturday is about saying to the government – and to anyone who wants to form the next government – do your policies measure up?” says James.

These concrete political aims are an important way in which the movement differs from last weekend’s March For Science. That protest, inspired by the popularity of the Women’s March earlier this year, kept its message intentionally wide. As one of the London event’s organisers told DeSmog, it placed its emphasis on a generalised “celebration of science”. But this lack of specificity drew criticism from some quarters – for presenting a confusing message about politics' relationship to science.

Generalisation can also risk putting people off joining marches at all. Over the last few months, numerous friends have said they feel uncomfortable joining protests where they’re not sure that the person marching next to them is doing so for the same reasons. They’d feel much happier signing a petition, with a more specific and limited aim, they tell me.

This weekend’s climate marches risk drawing some of the same concerns. “Climate-change has become a synecdoche, a surrogate, for many causes in today’s world – social justice, the protection of nature, the rights of future generations, the defence of science,” says Professor Mike Hulme from King's College London. “Marches such as this give political voice to anti-establishment protest, but they don’t stop the climate changing.”

In addition, not all who want to see climate change prioritised by governments may agree over the exact course of action – with outright opposition to fracking, for instance, or to a third runway at Heathrow.

But this weekend’s movement also appears to have taken these lessons on board. First, they are putting their political aims up front. According the US event’s website, whereas the March for Science strove to be non-political, this movement “believes strongly in the need to call out the politicians.”

The link to the Paris Climate Treaty is helpful in this respect. The People’s Climate Movement traces its birth back to September 21 2014, the eve of the UN climate summit, when 400,000 people marched through New York demanding action on the climate crisis. This gives the movement a clear piece of legislation to both celebrate and defend.

And the London-based event is also attempting to re-think and expand what street-protests can achieve. “We’re doing a smaller action rather than a big march,” explains Claire James, “but we’re trying to have a real focus with the speakers on ‘what next’”. After the protest in Westminster, attendees are invited to join an afternoon of free food, activities and music, hosted by the food waste campaign Feedback. Here there will be even further opportunity to learn about the many ways – from divestment campaigns to local renewable energy groups – in which people can help press for change.

In this respect, public action against the climate crisis promises not to end when the walking does. And while protests won't stop climate change in themselves, joining a march can be a powerful reminder that we are not in this crisis alone.

India Bourke is an environment writer and editorial assistant at the New Statesman.

0800 7318496