The only people sadder than Ed Miliband? YouGov. Photo:Getty
Show Hide image

Twilight of the pollsters: what should the polling companies do next?

The polling industry is at a low ebb after the disastrous result on Thursday. What can they do next?

After every election comes a period of mournful soul-searching for those who lost. Unusually, however, this time the group of losers includes not just defeated parties and their crestfallen leaders, but also a class of people who normally remain untouched by the vagaries of political fortune: opinion pollsters. For better or (as it turned out) for worse, this has arguably been the first ‘opinion poll election’. More polls were conducted than ever before, in more granular detail than ever before, and more attention than ever before was paid to how each shift and shimmy might translate into parties’ Westminster seat shares. Coalition flirtations, threats, and accusations dominated the ‘short campaign’—driven almost exclusively by the apparent ‘truth’ revealed by the polls, that a hung parliament was inevitable. As the public opinion scholar Lindsay Rogers might have put it, vox pollsteri seemed to have become synonymous with vox populi, and (equally inexorably) vox dei.

And yet, just as in 2010, as what looked initially like a wildly implausible exit poll was proved agonisingly (for the left), spectacularly (for the right) accurate over the course of the night, weeks’ worth of simulations and poll aggregations were seemingly discredited in the blink of an eye. The British public delivered a result that wholly confounded (nearly) all predictions—and the polls went from deified to vilified in the space of less than 24 hours, while John Curtice and his team of exit pollsters reigned supreme. Worse, that most sacred of things, the pollsters’ professional neutrality, was called seriously into question, and the British Polling Council took the remarkable step of setting up an independent enquiry into structural bias towards Labour and against the Conservatives within the industry. By any lights, a truly dramatic reversal—and one that raises the very real threat of a Pollsterdämmerung, a ‘twilight of the pollsters’.

In the corridors of power, the knives are out—as they already were well before the election—with indignant elites arming for a showdown with the polling industry. As in 1992, there have been calls to ban polls in the weeks before polling day, with the added sting of corruption accusations thrown in for good measure. It goes without saying that restricting opinion polling in this way is an overreaction, and an extremely dangerous route to take. Polls are a force for accountability in a democracy, as a regular reminder of the immense power that lies in the population, and of the political weight which its endorsement or rejection of an issue, party, or candidate carries. Without them, the ways to keep government informed of public opinion are reduced to rallies and protests, the media, or occasional fortuitous moments of personal contact. So rather than seeking to remove the influence of polls from our electoral cycle—which only benefits the powerful—the priority (as in 1992) should be to fix their problems. And the good news is that there are plenty of options for doing so.

One of these is what opinion researchers call ‘priming’, where pollsters ask the people they survey to think about specific important factors before asking them their voting intention. It is an inconvenient truth among both academics and professional pollsters that while polls can try to predict voting behaviour, they can never hope to explain it. This means that when polling companies contact their samples, their questions are not supposed to find out why people vote the way they do, but instead replicate how the average person might think if they were in the voting booth at that very moment. The effectiveness of ‘priming’  may have been been shown by Labour’s private polling in the run-up to the election, which asked respondents to “think about the country, the economy, their top issues, the parties and the leaders” before saying how they would vote, and produced Conservative and Labour relative vote shares that were far closer to the eventual outcome. Opinion polls in future must use such approaches much more. The focus on party leaders is especially important, and not just because of the rise of TV debates. Party elites personify what parties stand for, ideologically and strategically—just compare the mental pictures conjured up by ‘Tony Blair’s Labour’ versus ‘Ed Miliband’s Labour’—which factors into voters’ decisions even when they are primarily choosing between candidates for their constituency MPs.

But a far more sweeping reform concerns how pollsters measure the level of support parties enjoy within the population. Nothing can replace the situation of actually being in a voting booth, so even the carefully-framed forced choice of ‘voting intentions’ may really reflect much vaguer feelings of sympathy or allegiance towards the parties. What is needed is a way to determine how ‘intense’ support for different parties is—going beyond questions of ‘likelihood to vote’ (code for notoriously asymmetric turnout between party supporters) and the controversial exclusion or reallocation of ‘don’t knows’. Rather than simply asking respondents whether they support the prompted parties, pollsters should include ‘attitude scales’—either ranging from ‘weak’ to ‘strong’, or expressed as a numerical scale—to allow people to qualify their stated support: ‘weak Liberal Democrat’, ‘strong UKIP’, ‘ out of 10 Green’, and so on. While this would not rule out a recurrence of the famous ‘shy Tory’ factor, it would allow pollsters to identify more accurately the number of ‘switherers’ whose support for their parties might not be entrenched enough to prevent them switching their vote in the last instance.

A further addition would make pollsters’ analysis of voters’ conviction even more sophisticated, covering the ‘degree’ to which they back particular parties. Apart from a few die-hard supporters, most people are open to voting for more than one party, depending on their circumstances. Yet the closest the polling industry has come to modelling this is by adding questions about how respondents voted at the previous election—as with the apparent dependence, well documented by Politicalbetting.com, of projected Labour support at this election on ‘2010 Liberal Democrats’. As well as their partisan attitude scales, voting intention surveys should also include some measurement of ‘additional party preference’: either ordinal, like a poll equivalent of AV—‘1st choice Green, 2nd choice SNP, 3rd choice Labour’—or as a percentage distribution—‘70% Conservative, 30% UKIP’. This would allow pollsters to work out not just how many ‘switherers’ there are, but also where these voters might switch to—information which party strategists, at the very least, would surely be grateful for.

Ultimately, it remains to be seen quite what recommendations the Polling Council’s enquiry will produce. At the very least, however, the 2015 election has been a warning shot for the polling industry, which now knows that it needs (yet again) to up its game. The position of opinion polls as a staple feature of our democracy is far too precious to be wasted by methodological corner-cutting, or to be lightly abandoned in the face of political hostility. Their role of clarifying to society its own collective views about the pressing affairs of the day is not easy to fulfil. They have doubtless suffered a setback for now—but their situation is far more hopeful than the current narrative about them would have us believe.

Getty
Show Hide image

Donald Tusk is merely calling out Tory hypocrisy on Brexit

And the President of the European Council has the upper hand. 

The pair of numbers that have driven the discussion about our future relationship with the EU since the referendum have been 48 to 52. 

"The majority have spoken", cry the Leavers. "It’s time to tell the EU what we want and get out." However, even as they push for triggering the process early next year, the President of the European Council Donald Tusk’s reply to a letter from Tory MPs, where he blamed British voters for the uncertain futures of expats, is a long overdue reminder that another pair of numbers will, from now on, dominate proceedings.

27 to 1.

For all the media speculation around Brexit in the past few months, over what kind of deal the government will decide to be seek from any future relationship, it is incredible just how little time and thought has been given to the fact that once Article 50 is triggered, we will effectively be negotiating with 27 other partners, not just one.

Of course some countries hold more sway than others, due to their relative economic strength and population, but one of the great equalising achievements of the EU is that all of its member states have a voice. We need look no further than the last minute objections from just one federal entity within Belgium last month over CETA, the huge EU-Canada trade deal, to be reminded how difficult and important it is to build consensus.

Yet the Tories are failing spectacularly to understand this.

During his short trip to Strasbourg last week, David Davis at best ignored, and at worse angered, many of the people he will have to get on-side to secure a deal. Although he did meet Michel Barnier, the senior negotiator for the European Commission, and Guy Verhofstadt, the European Parliament’s representative at the future talks, he did not meet any representatives from the key Socialist Group in the European Parliament, nor the Parliament’s President, nor the Chair of its Constitutional Committee which will advise the Parliament on whether to ratify any future Brexit deal.

In parallel, Boris Johnson, to nobody’s surprise any more, continues to blunder from one debacle to the next, the most recent of which was to insult the Italians with glib remarks about prosecco sales.

On his side, Liam Fox caused astonishment by claiming that the EU would have to pay compensation to third countries across the world with which it has trade deals, to compensate them for Britain no longer being part of the EU with which they had signed their agreements!

And now, Theresa May has been embarrassingly rebuffed in her clumsy attempt to strike an early deal directly with Angela Merkel over the future residential status of EU citizens living and working in Britain and UK citizens in Europe. 

When May was campaigning to be Conservative party leader and thus PM, to appeal to the anti-european Tories, she argued that the future status of EU citizens would have to be part of the ongoing negotiations with the EU. Why then, four months later, are Tory MPs so quick to complain and call foul when Merkel and Tusk take the same position as May held in July? 

Because Theresa May has reversed her position. Our EU partners’ position remains the same - no negotiations before Article 50 is triggered and Britain sets out its stall. Merkel has said she can’t and won’t strike a pre-emptive deal.  In any case, she cannot make agreements on behalf of France,Netherlands and Austria, all of who have their own imminent elections to consider, let alone any other EU member. 

The hypocrisy of Tory MPs calling on the European Commission and national governments to end "the anxiety and uncertainty for UK and EU citizens living in one another's territories", while at the same time having caused and fuelled that same anxiety and uncertainty, has been called out by Tusk. 

With such an astounding level of Tory hypocrisy, incompetence and inconsistency, is it any wonder that our future negotiating partners are rapidly losing any residual goodwill towards the UK?

It is beholden on Theresa May’s government to start showing some awareness of the scale of the enormous task ahead, if the UK is to have any hope of striking a Brexit deal that is anything less than disastrous for Britain. The way they are handling this relatively simple issue does not augur well for the far more complex issues, involving difficult choices for Britain, that are looming on the horizon.

Richard Corbett is the Labour MEP for Yorkshire & Humber.