The only people sadder than Ed Miliband? YouGov. Photo:Getty
Show Hide image

Twilight of the pollsters: what should the polling companies do next?

The polling industry is at a low ebb after the disastrous result on Thursday. What can they do next?

After every election comes a period of mournful soul-searching for those who lost. Unusually, however, this time the group of losers includes not just defeated parties and their crestfallen leaders, but also a class of people who normally remain untouched by the vagaries of political fortune: opinion pollsters. For better or (as it turned out) for worse, this has arguably been the first ‘opinion poll election’. More polls were conducted than ever before, in more granular detail than ever before, and more attention than ever before was paid to how each shift and shimmy might translate into parties’ Westminster seat shares. Coalition flirtations, threats, and accusations dominated the ‘short campaign’—driven almost exclusively by the apparent ‘truth’ revealed by the polls, that a hung parliament was inevitable. As the public opinion scholar Lindsay Rogers might have put it, vox pollsteri seemed to have become synonymous with vox populi, and (equally inexorably) vox dei.

And yet, just as in 2010, as what looked initially like a wildly implausible exit poll was proved agonisingly (for the left), spectacularly (for the right) accurate over the course of the night, weeks’ worth of simulations and poll aggregations were seemingly discredited in the blink of an eye. The British public delivered a result that wholly confounded (nearly) all predictions—and the polls went from deified to vilified in the space of less than 24 hours, while John Curtice and his team of exit pollsters reigned supreme. Worse, that most sacred of things, the pollsters’ professional neutrality, was called seriously into question, and the British Polling Council took the remarkable step of setting up an independent enquiry into structural bias towards Labour and against the Conservatives within the industry. By any lights, a truly dramatic reversal—and one that raises the very real threat of a Pollsterdämmerung, a ‘twilight of the pollsters’.

In the corridors of power, the knives are out—as they already were well before the election—with indignant elites arming for a showdown with the polling industry. As in 1992, there have been calls to ban polls in the weeks before polling day, with the added sting of corruption accusations thrown in for good measure. It goes without saying that restricting opinion polling in this way is an overreaction, and an extremely dangerous route to take. Polls are a force for accountability in a democracy, as a regular reminder of the immense power that lies in the population, and of the political weight which its endorsement or rejection of an issue, party, or candidate carries. Without them, the ways to keep government informed of public opinion are reduced to rallies and protests, the media, or occasional fortuitous moments of personal contact. So rather than seeking to remove the influence of polls from our electoral cycle—which only benefits the powerful—the priority (as in 1992) should be to fix their problems. And the good news is that there are plenty of options for doing so.

One of these is what opinion researchers call ‘priming’, where pollsters ask the people they survey to think about specific important factors before asking them their voting intention. It is an inconvenient truth among both academics and professional pollsters that while polls can try to predict voting behaviour, they can never hope to explain it. This means that when polling companies contact their samples, their questions are not supposed to find out why people vote the way they do, but instead replicate how the average person might think if they were in the voting booth at that very moment. The effectiveness of ‘priming’  may have been been shown by Labour’s private polling in the run-up to the election, which asked respondents to “think about the country, the economy, their top issues, the parties and the leaders” before saying how they would vote, and produced Conservative and Labour relative vote shares that were far closer to the eventual outcome. Opinion polls in future must use such approaches much more. The focus on party leaders is especially important, and not just because of the rise of TV debates. Party elites personify what parties stand for, ideologically and strategically—just compare the mental pictures conjured up by ‘Tony Blair’s Labour’ versus ‘Ed Miliband’s Labour’—which factors into voters’ decisions even when they are primarily choosing between candidates for their constituency MPs.

But a far more sweeping reform concerns how pollsters measure the level of support parties enjoy within the population. Nothing can replace the situation of actually being in a voting booth, so even the carefully-framed forced choice of ‘voting intentions’ may really reflect much vaguer feelings of sympathy or allegiance towards the parties. What is needed is a way to determine how ‘intense’ support for different parties is—going beyond questions of ‘likelihood to vote’ (code for notoriously asymmetric turnout between party supporters) and the controversial exclusion or reallocation of ‘don’t knows’. Rather than simply asking respondents whether they support the prompted parties, pollsters should include ‘attitude scales’—either ranging from ‘weak’ to ‘strong’, or expressed as a numerical scale—to allow people to qualify their stated support: ‘weak Liberal Democrat’, ‘strong UKIP’, ‘ out of 10 Green’, and so on. While this would not rule out a recurrence of the famous ‘shy Tory’ factor, it would allow pollsters to identify more accurately the number of ‘switherers’ whose support for their parties might not be entrenched enough to prevent them switching their vote in the last instance.

A further addition would make pollsters’ analysis of voters’ conviction even more sophisticated, covering the ‘degree’ to which they back particular parties. Apart from a few die-hard supporters, most people are open to voting for more than one party, depending on their circumstances. Yet the closest the polling industry has come to modelling this is by adding questions about how respondents voted at the previous election—as with the apparent dependence, well documented by, of projected Labour support at this election on ‘2010 Liberal Democrats’. As well as their partisan attitude scales, voting intention surveys should also include some measurement of ‘additional party preference’: either ordinal, like a poll equivalent of AV—‘1st choice Green, 2nd choice SNP, 3rd choice Labour’—or as a percentage distribution—‘70% Conservative, 30% UKIP’. This would allow pollsters to work out not just how many ‘switherers’ there are, but also where these voters might switch to—information which party strategists, at the very least, would surely be grateful for.

Ultimately, it remains to be seen quite what recommendations the Polling Council’s enquiry will produce. At the very least, however, the 2015 election has been a warning shot for the polling industry, which now knows that it needs (yet again) to up its game. The position of opinion polls as a staple feature of our democracy is far too precious to be wasted by methodological corner-cutting, or to be lightly abandoned in the face of political hostility. Their role of clarifying to society its own collective views about the pressing affairs of the day is not easy to fulfil. They have doubtless suffered a setback for now—but their situation is far more hopeful than the current narrative about them would have us believe.

Show Hide image

We're racing towards another private debt crisis - so why did no one see it coming?

The Office for Budget Responsibility failed to foresee the rise in household debt. 

This is a call for a public inquiry on the current situation regarding private debt.

For almost a decade now, since 2007, we have been living a lie. And that lie is preparing to wreak havoc on our economy. If we do not create some kind of impartial forum to discuss what is actually happening, the results might well prove disastrous. 

The lie I am referring to is the idea that the financial crisis of 2008, and subsequent “Great Recession,” were caused by profligate government spending and subsequent public debt. The exact opposite is in fact the case. The crash happened because of dangerously high levels of private debt (a mortgage crisis specifically). And - this is the part we are not supposed to talk about—there is an inverse relation between public and private debt levels.

If the public sector reduces its debt, overall private sector debt goes up. That's what happened in the years leading up to 2008. Now austerity is making it happening again. And if we don't do something about it, the results will, inevitably, be another catastrophe.

The winners and losers of debt

These graphs show the relationship between public and private debt. They are both forecasts from the Office for Budget Responsibility, produced in 2015 and 2017. 

This is what the OBR was projecting what would happen around now back in 2015:

This year the OBR completely changed its forecast. This is how it now projects things are likely to turn out:

First, notice how both diagrams are symmetrical. What happens on top (that part of the economy that is in surplus) precisely mirrors what happens in the bottom (that part of the economy that is in deficit). This is called an “accounting identity.”

As in any ledger sheet, credits and debits have to match. The easiest way to understand this is to imagine there are just two actors, government, and the private sector. If the government borrows £100, and spends it, then the government has a debt of £100. But by spending, it has injected £100 more pounds into the private economy. In other words, -£100 for the government, +£100 for everyone else in the diagram. 

Similarly, if the government taxes someone for £100 , then the government is £100 richer but there’s £100 subtracted from the private economy (+£100 for government, -£100 for everybody else on the diagram).

So what implications does this kind of bookkeeping have for the overall economy? It means that if the government goes into surplus, then everyone else has to go into debt.

We tend to think of money as if it is a bunch of poker chips already lying around, but that’s not how it really works. Money has to be created. And money is created when banks make loans. Either the government borrows money and injects it into the economy, or private citizens borrow money from banks. Those banks don’t take the money from people’s savings or anywhere else, they just make it up. Anyone can write an IOU. But only banks are allowed to issue IOUs that the government will accept in payment for taxes. (In other words, there actually is a magic money tree. But only banks are allowed to use it.)

There are other factors. The UK has a huge trade deficit (blue), and that means the government (yellow) also has to run a deficit (print money, or more accurately, get banks to do it) to inject into the economy to pay for all those Chinese trainers, American iPads, and German cars. The total amount of money can also fluctuate. But the real point here is, the less the government is in debt, the more everyone else must be. Austerity measures will necessarily lead to rising levels of private debt. And this is exactly what has happened.

Now, if this seems to have very little to do with the way politicians talk about such matters, there's a simple reason: most politicians don’t actually know any of this. A recent survey showed 90 per cent of MPs don't even understand where money comes from (they think it's issued by the Royal Mint). In reality, debt is money. If no one owed anyone anything at all there would be no money and the economy would grind to a halt.

But of course debt has to be owed to someone. These charts show who owes what to whom.

The crisis in private debt

Bearing all this in mind, let's look at those diagrams again - keeping our eye particularly on the dark blue that represents household debt. In the first, 2015 version, the OBR duly noted that there was a substantial build-up of household debt in the years leading up to the crash of 2008. This is significant because it was the first time in British history that total household debts were higher than total household savings, and therefore the household sector itself was in deficit territory. (Corporations, at the same time, were raking in enormous profits.) But it also predicted this wouldn't happen again.

True, the OBR observed, austerity and the reduction of government deficits meant private debt levels would have to go up. However, the OBR economists insisted this wouldn't be a problem because the burden would fall not on households but on corporations. Business-friendly Tory policies would, they insisted, inspire a boom in corporate expansion, which would mean frenzied corporate borrowing (that huge red bulge below the line in the first diagram, which was supposed to eventually replace government deficits entirely). Ordinary households would have little or nothing to worry about.

This was total fantasy. No such frenzied boom took place.

In the second diagram, two years later, the OBR is forced to acknowledge this. Corporations are just raking in the profits and sitting on them. The household sector, on the other hand, is a rolling catastrophe. Austerity has meant falling wages, less government spending on social services (or anything else), and higher de facto taxes. This puts the squeeze on household budgets and people are forced to borrow. As a result, not only are households in overall deficit for the second time in British history, the situation is actually worse than it was in the years leading up to 2008.

And remember: it was a mortgage crisis that set off the 2008 crash, which almost destroyed the world economy and plunged millions into penury. Not a crisis in public debt. A crisis in private debt.

An inquiry

In 2015, around the time the original OBR predictions came out, I wrote an essay in the Guardian predicting that austerity and budget-balancing would create a disastrous crisis in private debt. Now it's so clearly, unmistakably, happening that even the OBR cannot deny it.

I believe the time has come for there be a public investigation - a formal public inquiry, in fact - into how this could be allowed to happen. After the 2008 crash, at least the economists in Treasury and the Bank of England could plausibly claim they hadn't completely understood the relation between private debt and financial instability. Now they simply have no excuse.

What on earth is an institution called the “Office for Budget Responsibility” credulously imagining corporate borrowing binges in order to suggest the government will balance the budget to no ill effects? How responsible is that? Even the second chart is extremely odd. Up to 2017, the top and bottom of the diagram are exact mirrors of one another, as they ought to be. However, in the projected future after 2017, the section below the line is much smaller than the section above, apparently seriously understating the amount both of future government, and future private, debt. In other words, the numbers don't add up.

The OBR told the New Statesman ​that it was not aware of any errors in its 2015 forecast for corporate sector net lending, and that the forecast was based on the available data. It said the forecast for business investment has been revised down because of the uncertainty created by Brexit. 

Still, if the “Office of Budget Responsibility” was true to its name, it should be sounding off the alarm bells right about now. So far all we've got is one mention of private debt and a mild warning about the rise of personal debt from the Bank of England, which did not however connect the problem to austerity, and one fairly strong statement from a maverick columnist in the Daily Mail. Otherwise, silence. 

The only plausible explanation is that institutions like the Treasury, OBR, and to a degree as well the Bank of England can't, by definition, warn against the dangers of austerity, however alarming the situation, because they have been set up the way they have in order to justify austerity. It's important to emphasise that most professional economists have never supported Conservative policies in this regard. The policy was adopted because it was convenient to politicians; institutions were set up in order to support it; economists were hired in order to come up with arguments for austerity, rather than to judge whether it would be a good idea. At present, this situation has led us to the brink of disaster.

The last time there was a financial crash, the Queen famously asked: why was no one able to foresee this? We now have the tools. Perhaps the most important task for a public inquiry will be to finally ask: what is the real purpose of the institutions that are supposed to foresee such matters, to what degree have they been politicised, and what would it take to turn them back into institutions that can at least inform us if we're staring into the lights of an oncoming train?