Photo: Getty
Show Hide image

Is polling dead?

The fault is not in our pollsters, but how the media covers them. 

Donald Trump is President-Elect of the United States. Britain is leaving the European Union. Ed Miliband’s not Prime Minister. The past eighteen months have left the reputation of both pollsters and poll aggregators in tatters. Failure of the polls in Britain led to calls for regulation. There is talk they should be abandoned altogether in favour of alternative sources of data, or that journalists’ time would be better spent going to speak to people in their own communities. 

The polling misses in the United Kingdom and United States are damaging not just because they have misled people about the likely outcome of the race. They also add fuel on the fire in an age in which experts are mocked. Pollsters – and the pundits that use them – are yet another elite to be put back in their place. To populists, the unpredictability of voters is another feather in their cap. Can polling survive its recent troubles?

There should be no doubt that the vast majority of pollsters are highly skilled and diligent in what they do. Every pollster that I have ever met cares about getting the election result right. Even setting aside motives of professional honour, there are overriding commercial incentives for polling firms to not go down in flames in the most publicly visible showcasing of their product.

Polling is getting harder. Pollsters face declining response rates to their surveys, with respondents increasingly hard to reach. Electorates across many advanced democracies are also undergoing substantial changes – with voters becoming more volatile and more detached from the parties that they have traditionally supported, and with issue and identity-based politics giving rise to new electoral coalitions. This creates a fast-moving target for pollsters from election to election. Yet there is little empirical evidence that polling errors are actually larger today than they used to be (if anything, data that I have collected of nearly 30,000 polls in 45 countries since the 1940s seems to indicate the reverse).

In polling for the presidential election, the national polls were actually only off by a couple of points and correctly picked Hillary Clinton as the winner of the popular vote (some political science forecasts also pointed to a close race or a Trump victory). What mattered was the misses in state polling (which had the added effect of making the forecasts of poll aggregators seem much more certain than they should have been).

But errors were not evenly spread across the battleground states. The more whites without degrees in a state, the greater the under-estimation of Trump’s share of the vote. This points to the turnout models having been badly wrong in predicting who would vote. The problem was that errors in state polling contributed to over-confident predictions of the outcome by poll aggregators. But this was not the pollsters’ fault, just as the Brexit polls often showed Leave in the lead but this was overlooked by many commentators.

Throwing in the towel is not the solution. At a time when our politics is increasingly fractured and ghettoised, by geography and in filter bubbles online, we need systematic measures of public opinion. Vox pops on street corners or focus groups may provide us rich insights into the way in which small numbers of voters are thinking about issues. But how can we know these attitudes extend to the public more generally? Indeed, survey research, including commercial polling, has pointed to many of the trends underlying Brexit and the rise of Trump for some time.

All polls are subject to sampling error and potential for bias. This makes the estimation of precise ‘point estimates’ of a vote percentage much more difficult – and this is where things are more likely to go wrong. Yet news coverage dominantly favours reporting of stories about who is up or down in the polls. Demand for this sort of polling is driven by media and their readers or viewers. Yet most polls are presented without caveat. Even the widely referred to ‘margin of error’ attached to polls is misleading. The margin of error is only applicable to random sampling error, and none of the opinion polls reported in the media today could be described as pure random probability polls (they are typically stratified samples of some sort). Instead, error is a function of the variety of weighting adjustments that are made to poll samples. Often the distribution of uncertainty around polls is not calculated. The tendency of commentators to talk about margin of error means that other sources of error and bias are often overlooked. It is not pollsters’ fault that there is such widespread ignorance about the elementary statistical theory behind polling. If polls were digested and reported in a more careful way, there would not be such surprise when they go wrong. Of course they would make for less interesting news stories too.

Polls still provide us with valuable information, if we can accept that they sometimes may be off by a matter of a few percentage points. Polling itself should have encouraged much more scepticism about the outcome of the presidential election. Thanks to the polls, everyone knew this was not a normal race. Trump and Clinton had the highest unfavourable ratings of any candidates in history. This alone should have suggested that normal rules might not apply, or that the behaviour of voters would be less predictable than usual.

Blaming pollsters is the easy way out. We need to become better, more informed and more critical consumers of polls.

Will Jennings is Professor of Political Science and Public Policy at the University of Southampton

Photo: Getty
Show Hide image

The rise of the green mayor – Sadiq Khan and the politics of clean energy

At an event at Tate Modern, Sadiq Khan pledged to clean up London's act.

On Thursday night, deep in the bowls of Tate Modern’s turbine hall, London Mayor Sadiq Khan renewed his promise to make the capital a world leader in clean energy and air. Yet his focus was as much on people as power plants – in particular, the need for local authorities to lead where central governments will not.

Khan was there to introduce the screening of a new documentary, From the Ashes, about the demise of the American coal industry. As he noted, Britain continues to battle against the legacy of fossil fuels: “In London today we burn very little coal but we are facing new air pollution challenges brought about for different reasons." 

At a time when the world's leaders are struggling to keep international agreements on climate change afloat, what can mayors do? Khan has pledged to buy only hybrid and zero-emissions buses from next year, and is working towards London becoming a zero carbon city.

Khan has, of course, also gained heroic status for being a bête noire of climate-change-denier-in-chief Donald Trump. On the US president's withdrawal from the Paris Agreement, Khan quipped: “If only he had withdrawn from Twitter.” He had more favourable things to say about the former mayor of New York and climate change activist Michael Bloomberg, who Khan said hailed from “the second greatest city in the world.”

Yet behind his humour was a serious point. Local authorities are having to pick up where both countries' central governments are leaving a void – in improving our air and supporting renewable technology and jobs. Most concerning of all, perhaps, is the way that interest groups representing business are slashing away at the regulations which protect public health, and claiming it as a virtue.

In the UK, documents leaked to Greenpeace’s energy desk show that a government-backed initiative considered proposals for reducing EU rules on fire-safety on the very day of the Grenfell Tower fire. The director of this Red Tape Initiative, Nick Tyrone, told the Guardian that these proposals were rejected. Yet government attempts to water down other EU regulations, such as the energy efficiency directive, still stand.

In America, this blame-game is even more highly charged. Republicans have sworn to replace what they describe as Obama’s “war on coal” with a war on regulation. “I am taking historic steps to lift the restrictions on American energy, to reverse government intrusion, and to cancel job-killing regulations,” Trump announced in March. While he has vowed “to promote clean air and clear water,” he has almost simultaneously signed an order to unravel the Clean Water Rule.

This rhetoric is hurting the very people it claims to protect: miners. From the Ashes shows the many ways that the industry harms wider public health, from water contamination, to air pollution. It also makes a strong case that the American coal industry is in terminal decline, regardless of possibile interventions from government or carbon capture.

Charities like Bloomberg can only do so much to pick up the pieces. The foundation, which helped fund the film, now not only helps support job training programs in coal communities after the Trump administration pulled their funding, but in recent weeks it also promised $15m to UN efforts to tackle climate change – again to help cover Trump's withdrawal from Paris Agreement. “I'm a bit worried about how many cards we're going to have to keep adding to the end of the film”, joked Antha Williams, a Bloomberg representative at the screening, with gallows humour.

Hope also lies with local governments and mayors. The publication of the mayor’s own environment strategy is coming “soon”. Speaking in panel discussion after the film, his deputy mayor for environment and energy, Shirley Rodrigues, described the move to a cleaner future as "an inevitable transition".

Confronting the troubled legacies of our fossil fuel past will not be easy. "We have our own experiences here of our coal mining communities being devastated by the closure of their mines," said Khan. But clean air begins with clean politics; maintaining old ways at the price of health is not one any government must pay. 

'From The Ashes' will premiere on National Geograhpic in the United Kingdom at 9pm on Tuesday, June 27th.

India Bourke is an environment writer and editorial assistant at the New Statesman.

0800 7318496