View all newsletters
Sign up to our newsletters

Support 110 years of independent journalism.

  1. Science & Tech
15 November 2012

Political middlemen and dart-throwing chimps

Martha Gill's "Irrational Animals" column.

By Martha Gill

Predicting the weather was once quite an interesting profession, needing skill in reading the instruments, intuition in deciphering the skies and years of experience in putting it all together. Now it’s the kind of job Nick Cage’s character would be given in a heavy-handed satire of the American dream, possibly also starring Michael Caine. We don’t need these skilled individuals any more – computers do all that. We just need an algorithm and a mouthpiece.

And so to Nate Silver – one of the biggest winners of the US presidential election. As the race neared its end, becoming “too close to call”, with money and opinions frantically changing hands, the New York Times blogger was calmly and correctly predicting voter outcome in every single state. He had what others didn’t – a formula to convert polling information into probabilities – and it turned out to be dead-on. He was not alone in getting it right but he was among the few. Many failed spectacularly.

Here’s Newt Gingrich on Fox News on 25 October: “I believe the minimum result will be 53-47 [per cent] Romney, over 300 electoral votes, and the Republicans will pick up the Senate. I base that . . . on just years and years of experience.” And here’s the GOP strategist Karl Rove in the Wall Street Journal on 31 October: “It comes down to numbers. And in the final days of this presidential race, from polling data to early voting, they favour Mitt Romney.”

These were not small errors. These people were standing in pre-hurricane wind and predicting sunshine. Are pundits more often wrong than not, or was it just this particular election that threw them? And how often do the statistics spewed out by experts hit the mark? One study found a statistic for it.

Algorithm blues

In the 1980s, a psychologist called Philip Tetlock took a group of journalists, foreign policy experts and economists – 284 of them – and spent the next two decades bombarding them with questions: would the dotcom bubble burst? Would George Bush be re-elected? How would apartheid end?

Select and enter your email address Your weekly guide to the best writing on ideas, politics, books and culture every Saturday. The best way to sign up for The Saturday Read is via saturdayread.substack.com The New Statesman's quick and essential guide to the news and politics of the day. The best way to sign up for Morning Call is via morningcall.substack.com Our Thursday ideas newsletter, delving into philosophy, criticism, and intellectual history. The best way to sign up for The Salvo is via thesalvo.substack.com Stay up to date with NS events, subscription offers & updates. Weekly analysis of the shift to a new economy from the New Statesman's Spotlight on Policy team.
  • Administration / Office
  • Arts and Culture
  • Board Member
  • Business / Corporate Services
  • Client / Customer Services
  • Communications
  • Construction, Works, Engineering
  • Education, Curriculum and Teaching
  • Environment, Conservation and NRM
  • Facility / Grounds Management and Maintenance
  • Finance Management
  • Health - Medical and Nursing Management
  • HR, Training and Organisational Development
  • Information and Communications Technology
  • Information Services, Statistics, Records, Archives
  • Infrastructure Management - Transport, Utilities
  • Legal Officers and Practitioners
  • Librarians and Library Management
  • Management
  • Marketing
  • OH&S, Risk Management
  • Operations Management
  • Planning, Policy, Strategy
  • Printing, Design, Publishing, Web
  • Projects, Programs and Advisors
  • Property, Assets and Fleet Management
  • Public Relations and Media
  • Purchasing and Procurement
  • Quality Management
  • Science and Technical Research and Development
  • Security and Law Enforcement
  • Service Delivery
  • Sport and Recreation
  • Travel, Accommodation, Tourism
  • Wellbeing, Community / Social Services
Visit our privacy Policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications.
THANK YOU

After analysing 82,361 predictions, Tetlock found that his experts performed worse than random chance. In short, they could have been beaten by dart-throwing chimps.

The reason was confidence. Tetlock found that the more often pundits appeared on TV, the more likely they were to be wrong. Their strong opinions were causing them to ignore dissenting facts or explain them away, leaving them trapped, he said, in the cage of their preconceptions.

Now, semi-expert middlemen are being squeezed out as the focus shifts to minute data analysis. Silver is one of the winners of this change but on the losing side is a whole industry of political forecasters. And it’s not just true of politics. Finance has been moving that way for a while. In UBS’s recent swath of job cuts, at least one trader, David Gallers, was replaced with an algorithm.

Difficult times for the old school, but what of the new? Silver expressed his concerns to the Wall Street Journal: “You don’t want to influence the system you are trying to forecast.” Only one problem with the new machines, then – accuracy. They’re so good that they might start controlling the weather.

Content from our partners
Labour's health reforms can put patients first
Data science can help developers design future-proof infrastructure
How to tackle the UK's plastic pollution problem – with Coca-Cola

Select and enter your email address Your weekly guide to the best writing on ideas, politics, books and culture every Saturday. The best way to sign up for The Saturday Read is via saturdayread.substack.com The New Statesman's quick and essential guide to the news and politics of the day. The best way to sign up for Morning Call is via morningcall.substack.com Our Thursday ideas newsletter, delving into philosophy, criticism, and intellectual history. The best way to sign up for The Salvo is via thesalvo.substack.com Stay up to date with NS events, subscription offers & updates. Weekly analysis of the shift to a new economy from the New Statesman's Spotlight on Policy team.
  • Administration / Office
  • Arts and Culture
  • Board Member
  • Business / Corporate Services
  • Client / Customer Services
  • Communications
  • Construction, Works, Engineering
  • Education, Curriculum and Teaching
  • Environment, Conservation and NRM
  • Facility / Grounds Management and Maintenance
  • Finance Management
  • Health - Medical and Nursing Management
  • HR, Training and Organisational Development
  • Information and Communications Technology
  • Information Services, Statistics, Records, Archives
  • Infrastructure Management - Transport, Utilities
  • Legal Officers and Practitioners
  • Librarians and Library Management
  • Management
  • Marketing
  • OH&S, Risk Management
  • Operations Management
  • Planning, Policy, Strategy
  • Printing, Design, Publishing, Web
  • Projects, Programs and Advisors
  • Property, Assets and Fleet Management
  • Public Relations and Media
  • Purchasing and Procurement
  • Quality Management
  • Science and Technical Research and Development
  • Security and Law Enforcement
  • Service Delivery
  • Sport and Recreation
  • Travel, Accommodation, Tourism
  • Wellbeing, Community / Social Services
Visit our privacy Policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications.
THANK YOU