Political middlemen and dart-throwing chimps

Martha Gill's "Irrational Animals" column.

Predicting the weather was once quite an interesting profession, needing skill in reading the instruments, intuition in deciphering the skies and years of experience in putting it all together. Now it’s the kind of job Nick Cage’s character would be given in a heavy-handed satire of the American dream, possibly also starring Michael Caine. We don’t need these skilled individuals any more – computers do all that. We just need an algorithm and a mouthpiece.

And so to Nate Silver – one of the biggest winners of the US presidential election. As the race neared its end, becoming “too close to call”, with money and opinions frantically changing hands, the New York Times blogger was calmly and correctly predicting voter outcome in every single state. He had what others didn’t – a formula to convert polling information into probabilities – and it turned out to be dead-on. He was not alone in getting it right but he was among the few. Many failed spectacularly.

Here’s Newt Gingrich on Fox News on 25 October: “I believe the minimum result will be 53-47 [per cent] Romney, over 300 electoral votes, and the Republicans will pick up the Senate. I base that . . . on just years and years of experience.” And here’s the GOP strategist Karl Rove in the Wall Street Journal on 31 October: “It comes down to numbers. And in the final days of this presidential race, from polling data to early voting, they favour Mitt Romney.”

These were not small errors. These people were standing in pre-hurricane wind and predicting sunshine. Are pundits more often wrong than not, or was it just this particular election that threw them? And how often do the statistics spewed out by experts hit the mark? One study found a statistic for it.

Algorithm blues

In the 1980s, a psychologist called Philip Tetlock took a group of journalists, foreign policy experts and economists – 284 of them – and spent the next two decades bombarding them with questions: would the dotcom bubble burst? Would George Bush be re-elected? How would apartheid end?

After analysing 82,361 predictions, Tetlock found that his experts performed worse than random chance. In short, they could have been beaten by dart-throwing chimps.

The reason was confidence. Tetlock found that the more often pundits appeared on TV, the more likely they were to be wrong. Their strong opinions were causing them to ignore dissenting facts or explain them away, leaving them trapped, he said, in the cage of their preconceptions.

Now, semi-expert middlemen are being squeezed out as the focus shifts to minute data analysis. Silver is one of the winners of this change but on the losing side is a whole industry of political forecasters. And it’s not just true of politics. Finance has been moving that way for a while. In UBS’s recent swath of job cuts, at least one trader, David Gallers, was replaced with an algorithm.

Difficult times for the old school, but what of the new? Silver expressed his concerns to the Wall Street Journal: “You don’t want to influence the system you are trying to forecast.” Only one problem with the new machines, then – accuracy. They’re so good that they might start controlling the weather.

Newt Gingrich opining away on Fox News. Photograph: Getty Images

Martha Gill writes the weekly Irrational Animals column. You can follow her on Twitter here: @Martha_Gill.

This article first appeared in the 19 November 2012 issue of the New Statesman, The plot against the BBC

Getty
Show Hide image

“A disaster waiting to happen”: Can you trust the government to digitise your personal data?

Privacy and security experts warn against the lesser-scrutinised Part 5 of the Digital Economy Bill, claiming bulk data sharing could be vulnerable to hacks.

Last week, the government’s Digital Economy Bill hit the news because of a proposed ban on pornographic websites that didn’t comply with its planned age verification rules. The news was just the right amount of shocking and yes, sexy, to grab the nation’s attention, but in the meantime other parts of the Bill remained unscrutinised. A distinctly un-sexy aspect of the Bill – Part 5, “Digital Government” – aims to completely revolutionise the way your personal data is shared.

In essence, Part 5 allows the government to digitise your data and bulk-share it without informing you or asking for your permission. This data includes your birth, death, and marriage certificates, as well as information on your taxes, court appearances, benefits, student loans, and even parking tickets. If the Bill passes, your information will be shared with local councils, charities, and even businesses – initially, gas and electricity companies.

Today, the Bill will undergo its third reading in the House of Commons. Last Friday, 26 privacy experts wrote to the Daily Telegraph to call for Part 5 to be removed from the Bill due to the lack of technical and legal safeguards in place.

“It's horrid and it's complex and it's going to impact all of us,” says Renate Samson, the chief executive of Big Brother Watch, an organisation that scrutinises the government to protect individual privacy. Big Brother Watch was invited by the government to work on the Bill as part of the government’s Open Policy Making, but Samson feels it was ignored when discussing the need for strong safeguards in the Bill. “Holding civil registration documents in bulk and sharing them in bulk is without a doubt a data disaster waiting to happen.”

Samson and her team worry that the Bill does not do enough to protect our personal data. “They tell a little story in one of their documents about mothers being able to click and access their baby’s birth certificate instead of having to go and get a copy, which sounds brilliant except they haven’t defined how they’ll know the mother is who she says she is, and how she will know who she can trust on the other end,” she says. “In a perfect, idyllic utopia, it works, but it doesn’t take hacking into consideration.”

According to the National Audit Office, in 2014-15, there were 9,000 data breaches across government departments. The subsequent inquiries revealed that many officials did not know how to report a breach and there was not enough guidance for the authorities involved. “The government is already failing to look after our data,” says Samson. “Fundamentally [Part 5] will lead to data breaches. People’s data will get lost and we won't ever know how or why.”

Though the government denies it, there are additional fears that this digitisation of data is the beginning of an ID database, a policy that was scrapped in 2011. At the time, then-Home Office minister Damian Green said that ending the proposed National Identity Register demonstrated “the government’s commitment to scale back the power of the state and restore civil liberties”.

Whether or not a register is created, however, Samson and other privacy experts, as well as the British Medical Association, take issue with the fundamental justifications for bulk data sharing. “The reason that they've given for wanting to do all this is ‘wellbeing’, which is crap, frankly,” she says. “In the summer, the Scottish Parliament dropped the Named Person Scheme because the supreme court found that ‘wellbeing’ is simply not a strong enough reason to share people’s personal information. Of course they’re trying to do something great but they’re going about it in a really cack-handed fashion.”

One example of this is that the government intends to share your personal information with the Troubled Families programme to identify people who may be at risk. Although this is ostensibly positive, this information will also be used to determine anti-social behaviour. “On the one hand, they’re saying that they’ll make sure that families who need help will get it, but on the other, if it transpires that you’re noisy or you’re difficult on your estate, they will now share that data so you can have an Asbo.”

Fundamentally, then, although the aims of the Bill seem admirable, there are simply not enough safeguards and rules in place currently for it to safely become law. While this partially might be a simple error on the government’s part, Samson argues that the language of the Bill is “as open and broad and woolly as you can possibly imagine”, causing concern about how it might actually be used in practice. In theory, hundreds or thousands of businesses and authorities could have access to your data without your consent.

“No one is opposing the idea of data sharing,” says Samson, “But a) tell us why, b) keep us informed if you’re using our data, and c) let us control our data. That’s the only way this is all going to move forward.”

Amelia Tait is a technology and digital culture writer at the New Statesman.