Oli Scarff/Getty Images
Show Hide image

Harvesting the wisdom of the crowd

The “superforecasters” who show remarkable proficiency in answering big geopolitical questions - despite their lack of knowledge.

Can the wisdom of crowds make more accurate predictions than top intelligence experts? Philip E Tetlock, a University of Pennsylvania political scientist, set up the Good Judgment Project in 2011 in order to find out. Funded by the US government, the programme identified a group of “superforecasters” who, despite possessing no particular expertise nor access to any secret information, have shown remarkable proficiency in answering big geopolitical questions – such as whether Serbia would be granted European Union candidacy by the end of 2011, or whether Bashar al-Assad’s regime in Syria would survive in 2012.

Supers, as they are known, are those in the top 2 per cent of participants. Tetlock’s thesis is that by aggregating the guesses of supers, he can derive a probability that comes close to the truth. Although no one can know everything about a given topic, everyone knows a little about something. Tetlock argues that aggregation amplifies what people get right and cancels out what they get wrong.

It sounds too good to be true: a relatively simple approach that overturns decades of received wisdom, a little like the case studies in Freakonomics or Moneyball, but specific to intelligence. What is astonishing is that the approach has been shown to work, with participants achieving accuracy scores well above random chance and supposedly even beating CIA analysts (though the agency apparently refuses to confirm this).

In October, the supers held a public conference at the London School of Economics. Before it opened, I met Andreas, an Italian super who lives in London and works in finance. When I asked him which forecast he was most proud of, he told me that he had closely predicted the number of banks that would fail the EU “stress test” in 2014. But was this forecast his “best”? Rather than giving a quick answer, Andreas instinctively began to deconstruct my rephrased question, asking rhetorically what might be meant by “best”. According to Tetlock, the ability to break down questions is a key skill in becoming a super.

The methods used to collate judgements have varied since 2011 but one of the most successful has been to use a market, or “forecasting tournament”, which works much like a real stock market. Each super is given 50,000 units of a fictional currency known as “Inklings” to invest in various outcomes. If, say, you had bet on Assad surviving a year ago and the odds were then at 10 per cent, you would have won a big return.

What was clear as I sat in the conference room – listening to the likes of the former Ministry of Defence analyst Nick Hare speaking about forecasting defence issues, or the conference organiser Michael Story bemoaning the predictions of media pundits – was that I was surrounded by very smart people. Pablo, the super next to me, was the dean of MBA students at a major business school. One questioner identified himself as working for the Bank of England. Contrary to the promotional pitch of Tetlock’s recent book, Superforecasting: the Art and Science of Prediction, these are not, for the most part, ordinary people, but highly qualified, well-connected and well-read individuals.

One participant suggested that the questions the supers performed worst on were those in their own professional field. When it came to breaking down a question and engaging with it objectively, he claimed, he was unable to see the wood for the trees. Chris, a strategy management consultant who joined the programme in its second year, described this quality positively as “conscious incompetence” – in essence, having an awareness of what you do not know and being open to having your forecasts challenged by others so that together you can arrive at better judgements.

Wanda, a Spanish-speaking guest from Texas (something of a rarity at the conference: the demography of supers seems to be overwhelmingly white and male), told me that she had performed best on questions relating to Iran. To do so, however, she had to be dispassionate, something she found difficult because she has strong opinions about Barack Obama’s recent nuclear deal with Iran. Ironically, for a programme that focuses on contentious political issues, the one taboo among supers seems to be personal politics. When I asked another participant if he would discuss his voting patterns with his super colleagues, he said that to do so would hinder co-operation.

So, what are the chances that this method of intelligence gathering will be taken seriously beyond this select group? I’m not so sure, but I know the right people to ask.

This article first appeared in the 29 October 2015 issue of the New Statesman, Israel: the Third Intifada?

The Depths of Hell
Show Hide image

Review: “Self-Portrait as Hairless Dog”, Alex Jones, 2018

Jones’s burners are off, but his face shows no fear: here is a creature that would luxuriate in hellfire as if it were as pleasurable as a hot shower.

The self-portrait as self-abasement has a long history in art: Caravaggio gave his own face to the severed head of Goliath being held aloft by the young sword-wielding David; Stanley Spencer once depicted his sunken haunches, grey skin and squashed genitalia (alongside his wife’s sagging body) next to a carefully depicted leg of mutton; Michelangelo meanwhile showed himself as an empty flayed skin in The Last Judgement  in the Sistine Chapel.

What to make, then, of the mocked-up photograph currently scarring Twitter’s collective eyeballs which the radio host, conspiracy theorist and provocateur Alex Jones posted, portraying him as a hairless dog lying on a kitchen hob?

This strange, Hieronymus Bosch monster, stares expressionless at the viewer anticipating a moue of distaste. The Jones-hound is unapologetic, fleshily pink in a pose that carries uncomfortable references to Renaissance nudes.

Titian’s Venus D’Urbino shows a voluptuously sensual woman: the thoughts she is meant to evoke can only be carnal. But it is harder to see the Jones image, however coquettish, as drawing the panting male gaze. Is his nakedness a reminder that we are all born of original sin and creatures of shame, like Adam and Eve expelled from the Garden of Eden? Is his canine persona an acknowledgement that we nothing but animals?

The kitchen hob on which this squidgy hybrid lounges clearly alludes to the flames of hell. Rogier van der Weyden showed the flames licking at terrified sinners in his Last Judgement of 1450: this, he says, is what happens to those who err from the ways of the Bible. Jones’s burners are off, but his face shows no fear: here is a creature that would luxuriate in hellfire as if it were as pleasurable as a hot shower.

It is hard not to interpret the Caravaggio, Spencer and Michelangelo depictions other than as as expressions of self-loathing, of a disgust so profound it came to the fore almost despite themselves. Jones, though, looks complacent, even contented. Medieval bestiaries are full of such fanciful creatures; often they are emblems of evil, the Devil’s playthings that are beyond redemption and settled in their fallen state. This, perhaps, is closer to the truth.

However, the cleverness of the image ultimately lies in something it doesn’t show. The animal’s plump tail covers, almost coyly, its genitals and what Jones’s expression says is: don’t worry, if you are looking for bollocks, well, that’s me.

Michael Prodger is Reviews Editor at the New Statesman. He is an art historian, Senior Research Fellow at the University of Buckingham, and a former literary editor.