Crowdsourcing the unemployment rate

What's inflation going to be? Wanna bet?

"Wisdom of the crowds" is a pretty solid phenomenon. Ask a thousand people to guess the number of sweets in a jar, and the average (mean) of their guesses tends to be damn close to the actual number.

What's more interesting is whether the same idea works, not just to guesses, but to forecasts. Specifically, economic forecasts. If you ask a thousand people to guess what the unemployment rate will be in two years time, how will they do?

There are certainly reasons to be hopeful. Information is widely distributed, with little advantage accruing to experts; and in fact, unlike with simply counting sweets, there's likely to be a fair few people with "inside" information (hiring plans, perhaps, or a feel for how their sector is moving), which they may use to inform their guesses. Mix together enough guesses, and you could generate insight.

That's what the Adam Smith Institute and Paddy Power are hoping; the two have teamed up to offer markets in key UK economic statistics. You'll be able to bet on what the rate of inflation and unemployment will be in June 2015; the ASI's Sam Bowman writes that:

By combining the local knowledge of thousands of people, betting markets can outpredict any panel of experts. If these markets catch on, the government should consider outsourcing all of its forecasts to prediction markets instead of expert forecasters.

But there may still be some problems, both with the idea and its implementation.

Betting markets are indeed a theoretically great way of harnessing the wisdom of the crowds. As Bowman writes, the fact that people put money on their predictions means that more confident predictions are weighted higher, and vice versa. But the necessity of teaming up with a bookmaker to launch the idea means that there is a major distortion: the odds the bookie has set. Punters can get 7/2 that inflation will be greater than 5 per cent, and just 5/2 that it will be between 4.01 per cent and 5 per cent. That means that someone who thinks that inflation is most likely to be around 4.75 per cent may take advantage of the higher odds offered if they guess slightly higher. It also means that what Paddy Power think is most likely will skew the guesses.

A better version of the same idea would be to create a prediction market. The difference between the two is that in a market, the crowd takes the role of bookmaker as well as punter. The odds themselves get set procedurally, based purely on where people are betting, and so there's no chance of a bad guess on the bookies' part skewing the predictions.

But even if the market was designed to perfectly get the true thoughts of everyone in the crowd, there's still reasons to doubt that it can be that good at forecasting economic data.

There's quite a specific set of conditions which are required for crowdsourcing to work. James Surowiecki, who coined the phrase "wisdom of crowds", describes four: Diversity of opinion, independence of opinion, decentralisation of action, and aggregation of information. Of those, the one which is the most problematic in this case is independence. People's guesses aren't secret, and they affect others. That means you could end up seeing a circular mill, where everyone reinforces everyone else's beliefs to the extent that the crowdsourcing breaks down. Think: do you hold your beliefs about what might happen to the unemployment rate based on investigation of the primary data, or based on collation of expert analysis? If it's the latter, you'd be a net harm to the crowdsourcing, contributing largely to the flocking problem.

It would still be nice to get more financial bets. But that's mostly so that I could join in my sportier friends in having something where I feel like my expertise could win me a bit of cash; when it comes to actually trying to work out what will happen, we might have to stick with older methods.

Mark Carney. Photograph: Getty Images

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Photo: Getty
Show Hide image

The rise of the green mayor – Sadiq Khan and the politics of clean energy

At an event at Tate Modern, Sadiq Khan pledged to clean up London's act.

On Thursday night, deep in the bowls of Tate Modern’s turbine hall, London Mayor Sadiq Khan renewed his promise to make the capital a world leader in clean energy and air. Yet his focus was as much on people as power plants – in particular, the need for local authorities to lead where central governments will not.

Khan was there to introduce the screening of a new documentary, From the Ashes, about the demise of the American coal industry. As he noted, Britain continues to battle against the legacy of fossil fuels: “In London today we burn very little coal but we are facing new air pollution challenges brought about for different reasons." 

At a time when the world's leaders are struggling to keep international agreements on climate change afloat, what can mayors do? Khan has pledged to buy only hybrid and zero-emissions buses from next year, and is working towards London becoming a zero carbon city.

Khan has, of course, also gained heroic status for being a bête noire of climate-change-denier-in-chief Donald Trump. On the US president's withdrawal from the Paris Agreement, Khan quipped: “If only he had withdrawn from Twitter.” He had more favourable things to say about the former mayor of New York and climate change activist Michael Bloomberg, who Khan said hailed from “the second greatest city in the world.”

Yet behind his humour was a serious point. Local authorities are having to pick up where both countries' central governments are leaving a void – in improving our air and supporting renewable technology and jobs. Most concerning of all, perhaps, is the way that interest groups representing business are slashing away at the regulations which protect public health, and claiming it as a virtue.

In the UK, documents leaked to Greenpeace’s energy desk show that a government-backed initiative considered proposals for reducing EU rules on fire-safety on the very day of the Grenfell Tower fire. The director of this Red Tape Initiative, Nick Tyrone, told the Guardian that these proposals were rejected. Yet government attempts to water down other EU regulations, such as the energy efficiency directive, still stand.

In America, this blame-game is even more highly charged. Republicans have sworn to replace what they describe as Obama’s “war on coal” with a war on regulation. “I am taking historic steps to lift the restrictions on American energy, to reverse government intrusion, and to cancel job-killing regulations,” Trump announced in March. While he has vowed “to promote clean air and clear water,” he has almost simultaneously signed an order to unravel the Clean Water Rule.

This rhetoric is hurting the very people it claims to protect: miners. From the Ashes shows the many ways that the industry harms wider public health, from water contamination, to air pollution. It also makes a strong case that the American coal industry is in terminal decline, regardless of possibile interventions from government or carbon capture.

Charities like Bloomberg can only do so much to pick up the pieces. The foundation, which helped fund the film, now not only helps support job training programs in coal communities after the Trump administration pulled their funding, but in recent weeks it also promised $15m to UN efforts to tackle climate change – again to help cover Trump's withdrawal from Paris Agreement. “I'm a bit worried about how many cards we're going to have to keep adding to the end of the film”, joked Antha Williams, a Bloomberg representative at the screening, with gallows humour.

Hope also lies with local governments and mayors. The publication of the mayor’s own environment strategy is coming “soon”. Speaking in panel discussion after the film, his deputy mayor for environment and energy, Shirley Rodrigues, described the move to a cleaner future as "an inevitable transition".

Confronting the troubled legacies of our fossil fuel past will not be easy. "We have our own experiences here of our coal mining communities being devastated by the closure of their mines," said Khan. But clean air begins with clean politics; maintaining old ways at the price of health is not one any government must pay. 

'From The Ashes' will premiere on National Geograhpic in the United Kingdom at 9pm on Tuesday, June 27th.

India Bourke is an environment writer and editorial assistant at the New Statesman.

0800 7318496