Crowdsourcing the unemployment rate

What's inflation going to be? Wanna bet?

"Wisdom of the crowds" is a pretty solid phenomenon. Ask a thousand people to guess the number of sweets in a jar, and the average (mean) of their guesses tends to be damn close to the actual number.

What's more interesting is whether the same idea works, not just to guesses, but to forecasts. Specifically, economic forecasts. If you ask a thousand people to guess what the unemployment rate will be in two years time, how will they do?

There are certainly reasons to be hopeful. Information is widely distributed, with little advantage accruing to experts; and in fact, unlike with simply counting sweets, there's likely to be a fair few people with "inside" information (hiring plans, perhaps, or a feel for how their sector is moving), which they may use to inform their guesses. Mix together enough guesses, and you could generate insight.

That's what the Adam Smith Institute and Paddy Power are hoping; the two have teamed up to offer markets in key UK economic statistics. You'll be able to bet on what the rate of inflation and unemployment will be in June 2015; the ASI's Sam Bowman writes that:

By combining the local knowledge of thousands of people, betting markets can outpredict any panel of experts. If these markets catch on, the government should consider outsourcing all of its forecasts to prediction markets instead of expert forecasters.

But there may still be some problems, both with the idea and its implementation.

Betting markets are indeed a theoretically great way of harnessing the wisdom of the crowds. As Bowman writes, the fact that people put money on their predictions means that more confident predictions are weighted higher, and vice versa. But the necessity of teaming up with a bookmaker to launch the idea means that there is a major distortion: the odds the bookie has set. Punters can get 7/2 that inflation will be greater than 5 per cent, and just 5/2 that it will be between 4.01 per cent and 5 per cent. That means that someone who thinks that inflation is most likely to be around 4.75 per cent may take advantage of the higher odds offered if they guess slightly higher. It also means that what Paddy Power think is most likely will skew the guesses.

A better version of the same idea would be to create a prediction market. The difference between the two is that in a market, the crowd takes the role of bookmaker as well as punter. The odds themselves get set procedurally, based purely on where people are betting, and so there's no chance of a bad guess on the bookies' part skewing the predictions.

But even if the market was designed to perfectly get the true thoughts of everyone in the crowd, there's still reasons to doubt that it can be that good at forecasting economic data.

There's quite a specific set of conditions which are required for crowdsourcing to work. James Surowiecki, who coined the phrase "wisdom of crowds", describes four: Diversity of opinion, independence of opinion, decentralisation of action, and aggregation of information. Of those, the one which is the most problematic in this case is independence. People's guesses aren't secret, and they affect others. That means you could end up seeing a circular mill, where everyone reinforces everyone else's beliefs to the extent that the crowdsourcing breaks down. Think: do you hold your beliefs about what might happen to the unemployment rate based on investigation of the primary data, or based on collation of expert analysis? If it's the latter, you'd be a net harm to the crowdsourcing, contributing largely to the flocking problem.

It would still be nice to get more financial bets. But that's mostly so that I could join in my sportier friends in having something where I feel like my expertise could win me a bit of cash; when it comes to actually trying to work out what will happen, we might have to stick with older methods.

Mark Carney. Photograph: Getty Images

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Getty Images.
Show Hide image

Voters are turning against Brexit but the Lib Dems aren't benefiting

Labour's pro-Brexit stance is not preventing it from winning the support of Remainers. Will that change?

More than a year after the UK voted for Brexit, there has been little sign of buyer's remorse. The public, including around a third of Remainers, are largely of the view that the government should "get on with it".

But as real wages are squeezed (owing to the Brexit-linked inflationary spike) there are tentative signs that the mood is changing. In the event of a second referendum, an Opinium/Observer poll found, 47 per cent would vote Remain, compared to 44 per cent for Leave. Support for a repeat vote is also increasing. Forty one per cent of the public now favour a second referendum (with 48 per cent opposed), compared to 33 per cent last December. 

The Liberal Democrats have made halting Brexit their raison d'être. But as public opinion turns, there is no sign they are benefiting. Since the election, Vince Cable's party has yet to exceed single figures in the polls, scoring a lowly 6 per cent in the Opinium survey (down from 7.4 per cent at the election). 

What accounts for this disparity? After their near-extinction in 2015, the Lib Dems remain either toxic or irrelevant to many voters. Labour, by contrast, despite its pro-Brexit stance, has hoovered up Remainers (55 per cent back Jeremy Corbyn's party). 

In some cases, this reflects voters' other priorities. Remainers are prepared to support Labour on account of the party's stances on austerity, housing and education. Corbyn, meanwhile, is a eurosceptic whose internationalism and pro-migration reputation endear him to EU supporters. Other Remainers rewarded Labour MPs who voted against Article 50, rebelling against the leadership's stance. 

But the trend also partly reflects ignorance. By saying little on the subject of Brexit, Corbyn and Labour allowed Remainers to assume the best. Though there is little evidence that voters will abandon Corbyn over his EU stance, the potential exists.

For this reason, the proposal of a new party will continue to recur. By challenging Labour over Brexit, without the toxicity of Lib Dems, it would sharpen the choice before voters. Though it would not win an election, a new party could force Corbyn to soften his stance on Brexit or to offer a second referendum (mirroring Ukip's effect on the Conservatives).

The greatest problem for the project is that it lacks support where it counts: among MPs. For reasons of tribalism and strategy, there is no emergent "Gang of Four" ready to helm a new party. In the absence of a new convulsion, the UK may turn against Brexit without the anti-Brexiteers benefiting. 

George Eaton is political editor of the New Statesman.