Political middlemen and dart-throwing chimps

Martha Gill's "Irrational Animals" column.

Predicting the weather was once quite an interesting profession, needing skill in reading the instruments, intuition in deciphering the skies and years of experience in putting it all together. Now it’s the kind of job Nick Cage’s character would be given in a heavy-handed satire of the American dream, possibly also starring Michael Caine. We don’t need these skilled individuals any more – computers do all that. We just need an algorithm and a mouthpiece.

And so to Nate Silver – one of the biggest winners of the US presidential election. As the race neared its end, becoming “too close to call”, with money and opinions frantically changing hands, the New York Times blogger was calmly and correctly predicting voter outcome in every single state. He had what others didn’t – a formula to convert polling information into probabilities – and it turned out to be dead-on. He was not alone in getting it right but he was among the few. Many failed spectacularly.

Here’s Newt Gingrich on Fox News on 25 October: “I believe the minimum result will be 53-47 [per cent] Romney, over 300 electoral votes, and the Republicans will pick up the Senate. I base that . . . on just years and years of experience.” And here’s the GOP strategist Karl Rove in the Wall Street Journal on 31 October: “It comes down to numbers. And in the final days of this presidential race, from polling data to early voting, they favour Mitt Romney.”

These were not small errors. These people were standing in pre-hurricane wind and predicting sunshine. Are pundits more often wrong than not, or was it just this particular election that threw them? And how often do the statistics spewed out by experts hit the mark? One study found a statistic for it.

Algorithm blues

In the 1980s, a psychologist called Philip Tetlock took a group of journalists, foreign policy experts and economists – 284 of them – and spent the next two decades bombarding them with questions: would the dotcom bubble burst? Would George Bush be re-elected? How would apartheid end?

After analysing 82,361 predictions, Tetlock found that his experts performed worse than random chance. In short, they could have been beaten by dart-throwing chimps.

The reason was confidence. Tetlock found that the more often pundits appeared on TV, the more likely they were to be wrong. Their strong opinions were causing them to ignore dissenting facts or explain them away, leaving them trapped, he said, in the cage of their preconceptions.

Now, semi-expert middlemen are being squeezed out as the focus shifts to minute data analysis. Silver is one of the winners of this change but on the losing side is a whole industry of political forecasters. And it’s not just true of politics. Finance has been moving that way for a while. In UBS’s recent swath of job cuts, at least one trader, David Gallers, was replaced with an algorithm.

Difficult times for the old school, but what of the new? Silver expressed his concerns to the Wall Street Journal: “You don’t want to influence the system you are trying to forecast.” Only one problem with the new machines, then – accuracy. They’re so good that they might start controlling the weather.

Newt Gingrich opining away on Fox News. Photograph: Getty Images

Martha Gill writes the weekly Irrational Animals column. You can follow her on Twitter here: @Martha_Gill.

This article first appeared in the 19 November 2012 issue of the New Statesman, The plot against the BBC

Getty
Show Hide image

From Darwin to Damore - the ancient art of using "science" to mask prejudice

Charles Darwin, working at a time when women had little legal rights, declared “woman is a kind of adult child”.

“In addition to the Left’s affinity for those it sees as weak, humans are generally biased towards protecting females,” wrote James Damore, in his now infamous anti-diversity Google memo. “As mentioned before, this likely evolved because males are biologically disposable and because women are generally more co-operative and agreeable than men.” Since the memo was published, hordes of women have come forward to say that views like these – where individuals justify bias on the basis of science – are not uncommon in their traditionally male-dominated fields. Damore’s controversial screed set off discussions about the age old debate: do biological differences justify discrimination?  

Modern science developed in a society which assumed that man was superior over women. Charles Darwin, the father of modern evolutionary biology, who died before women got the right to vote, argued that young children of both genders resembled adult women more than they did adult men; as a result, “woman is a kind of adult child”.

Racial inequality wasn’t immune from this kind of theorising either. As fields such as psychology and genetics developed a greater understanding about the fundamental building blocks of humanity, many prominent researchers such as Francis Galton, Darwin’s cousin, argued that there were biological differences between races which explained the ability of the European race to prosper and gather wealth, while other races fell far behind. The same kind of reasoning fuelled the Nazi eugenics and continues to fuel the alt-right in their many guises today.

Once scorned as blasphemy, today "science" is approached by many non-practitioners with a cult-like reverence. Attributing the differences between races and gender to scientific research carries the allure of empiricism. Opponents of "diversity" would have you believe that scientific research validates racism and sexism, even though one's bleeding heart might wish otherwise. 

The problem is that current scientific research just doesn’t agree. Some branches of science, such as physics, are concerned with irrefutable laws of nature. But the reality, as evidenced by the growing convergence of social sciences like sociology, and life sciences, such as biology, is that science as a whole will, and should change. The research coming out of fields like genetics and psychology paint an increasingly complex picture of humanity. Saying (and proving) that gravity exists isn't factually equivalent to saying, and trying to prove, that women are somehow less capable at their jobs because of presumed inherent traits like submissiveness. 

When it comes to matters of race, the argument against racial realism, as it’s often referred to, is unequivocal. A study in 2002, authored by Neil Risch and others, built on the work of the Human Genome Project to examine the long standing and popular myth of seven distinct races. Researchers found that  “62 per cent of Ethiopians belong to the same cluster as Norwegians, together with 21 per cent of the Afro-Caribbeans, and the ethnic label ‘Asian’ inaccurately describes Chinese and Papuans who were placed almost entirely in separate clusters.” All that means is that white supremacists are wrong, and always have been.

Even the researcher Damore cites in his memo, Bradley Schmitt of Bradley University in Illinois, doesn’t agree with Damore’s conclusions.  Schmitt pointed out, in correspondence with Wired, that biological difference only accounts for about 10 per cent of the variance between men and women in what Damore characterises as female traits, such as neuroticism. In addition, nebulous traits such as being “people-oriented” are difficult to define and have led to wildly contradictory research from people who are experts in the fields. Suggesting that women are bad engineers because they’re neurotic is not only mildly ridiculous, but even unsubstantiated by Damore’s own research.  As many have done before him, Damore couched his own worldview - and what he was trying to convince others of - in the language of rationalism, but ultimately didn't pay attention to the facts.

And, even if you did buy into Damore's memo, a true scientist would retort - so what? It's a fallacy to argue that just because a certain state of affairs prevails, that that is the way that it ought to be. If that was the case, why does humanity march on in the direction of technological and industrial progress?

Humans weren’t meant to travel large distances, or we would possess the ability to do so intrinsically. Boats, cars, airplanes, trains, according to the Damore mindset, would be a perversion of nature. As a species, we consider overcoming biology to be a sign of success. 

Of course, the damage done by these kinds of views is not only that they’re hard to counteract, but that they have real consequences. Throughout history, appeals to the supposed rationalism of scientific research have justified moral atrocities such as ethnic sterilisation, apartheid, the creation of the slave trade, and state-sanctioned genocide.

If those in positions of power genuinely think that black and Hispanic communities are genetically predisposed to crime and murder, they’re very unlikely to invest in education, housing and community centres for those groups. Cycles of poverty then continue, and the myth, dressed up in pseudo-science, is entrenched. 

Damore and those like him will certainly maintain that the evidence for gender differences are on their side. Since he was fired from Google, Damore has become somewhat of an icon to some parts of society, giving interviews to right-wing Youtubers and posing in a dubious shirt parodying the Google logo (it now says Goolag). Never mind that Damore’s beloved science has already proved them wrong.