A statistical trick which reveals whether MPs are lying about expenses

Benford's law has many uses. Can it trip up MPs?

Are politicians routinely making up expenses? A simple statistical test suggests not.

Benford's law is a statistical artefact found in numerical data spanning several orders of magnitude. Ben Goldacre explains:

Imagine you have data on, say, the population of every world nation. Now, take only the "leading digit" from each number: the first number in the number, if you like. For the UK population, which was 61,838,154 in 2009, that leading digit would be "six". Andorra's was 85,168, so that's "eight". And so on.

If you take all those leading digits, from all the countries, then overall, you might naively expect to see the same number of ones, fours, nines, and so on. But in fact, for naturally occurring data, you get more ones than twos, more twos than threes, and so on, all the way down to nine. This is Benford's law: the distribution of leading digits follows a logarithmic distribution, so you get a "one" most commonly, appearing as first digit around 30% of the time, and a nine as first digit only 5% of the time.

This pattern should repeat for almost any data which matches the key condition of spanning a large range of sizes. Take the example above, world populations, which goes from 800 in the Vatican City to 1.35 billion in China. But one category of data which rarely obeys the law is that where the numbers are made-up. When people are trying to "randomly" write down numbers, they rarely do it very well, more frequently following the intuition that random data ought to have just as much chance of starting with any given digit.

The value of MP's expenses certainly spans several orders of magnitude. Excluding repaid claims, expenses in the latest tranche, released last week, span from a value of 10p (reconciliation for a travelcard between Euston and Coventry) to £9900 (for staffing costs in Woking constituency office).

So does the data follow Benford's law? It largely does:

 

The largest variation is a 3 percentage point difference between the expected number of leading 2s and the actual number, with most other digits being present in slightly larger quantities than expected.

Scanning through the data, it's easy to see why this is. There are a large number of claims which are made repeatedly. For instance, 18 different MPs claimed £139.26 for the same twin pack of HP toner cartridges; while nearly every claim for petrol costs came in between £10 and £19.99, boosting the 1s' count again. Conversely, there simply weren't that many must-have services which began with a 2 (although a lot of things MPs need do, apparently, cost £20 on the dot, from venue hire to cleaning bills and car parking).

None of which means there may not still be fraud in the expenses. It simply means that the actual values being claimed for have been drawn from real life. MPs are not, on the whole, making up numbers on the spot as the fill in expense forms; whether what they are claiming for ought to be paid out of the public pocket, statistics are less likely to help with.

(As an aside, it's actually surprising that the figures match Benford's law quite so well; while MP's may not be choosing the numbers they submit, the people who set the prices clearly are. That's probably the reason for the slight uptick in the 9s, for instance; a lot of things which may cost £10 instead are charged as £9.99. It seems that there are either enough counter-examples that it gets balanced out, or lots of claims for things like mileage, which have no set price)

Two data CDs, much like the ones which sparked the original expenses scandal. Photograph: Getty Images

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Getty.
Show Hide image

The Brexit Beartraps, #2: Could dropping out of the open skies agreement cancel your holiday?

Flying to Europe is about to get a lot more difficult.

So what is it this time, eh? Brexit is going to wipe out every banana planet on the entire planet? Brexit will get the Last Night of the Proms cancelled? Brexit will bring about World War Three?

To be honest, I think we’re pretty well covered already on that last score, but no, this week it’s nothing so terrifying. It’s just that Brexit might get your holiday cancelled.

What are you blithering about now?

Well, only if you want to holiday in Europe, I suppose. If you’re going to Blackpool you’ll be fine. Or Pakistan, according to some people...

You’re making this up.

I’m honestly not, though we can’t entirely rule out the possibility somebody is. Last month Michael O’Leary, the Ryanair boss who attracts headlines the way certain other things attract flies, warned that, “There is a real prospect... that there are going to be no flights between the UK and Europe for a period of weeks, months beyond March 2019... We will be cancelling people’s holidays for summer of 2019.”

He’s just trying to block Brexit, the bloody saboteur.

Well, yes, he’s been quite explicit about that, and says we should just ignore the referendum result. Honestly, he’s so Remainiac he makes me look like Dan Hannan.

But he’s not wrong that there are issues: please fasten your seatbelt, and brace yourself for some turbulence.

Not so long ago, aviation was a very national sort of a business: many of the big airports were owned by nation states, and the airline industry was dominated by the state-backed national flag carriers (British Airways, Air France and so on). Since governments set airline regulations too, that meant those airlines were given all sorts of competitive advantages in their own country, and pretty much everyone faced barriers to entry in others. 

The EU changed all that. Since 1994, the European Single Aviation Market (ESAM) has allowed free movement of people and cargo; established common rules over safety, security, the environment and so on; and ensured fair competition between European airlines. It also means that an AOC – an Air Operator Certificate, the bit of paper an airline needs to fly – from any European country would be enough to operate in all of them. 

Do we really need all these acronyms?

No, alas, we need more of them. There’s also ECAA, the European Common Aviation Area – that’s the area ESAM covers; basically, ESAM is the aviation bit of the single market, and ECAA the aviation bit of the European Economic Area, or EEA. Then there’s ESAA, the European Aviation Safety Agency, which regulates, well, you can probably guess what it regulates to be honest.

All this may sound a bit dry-

It is.

-it is a bit dry, yes. But it’s also the thing that made it much easier to travel around Europe. It made the European aviation industry much more competitive, which is where the whole cheap flights thing came from.

In a speech last December, Andrew Haines, the boss of Britain’s Civil Aviation Authority said that, since 2000, the number of destinations served from UK airports has doubled; since 1993, fares have dropped by a third. Which is brilliant.

Brexit, though, means we’re probably going to have to pull out of these arrangements.

Stop talking Britain down.

Don’t tell me, tell Brexit secretary David Davis. To monitor and enforce all these international agreements, you need an international court system. That’s the European Court of Justice, which ministers have repeatedly made clear that we’re leaving.

So: last March, when Davis was asked by a select committee whether the open skies system would persist, he replied: “One would presume that would not apply to us” – although he promised he’d fight for a successor, which is very reassuring. 

We can always holiday elsewhere. 

Perhaps you can – O’Leary also claimed (I’m still not making this up) that a senior Brexit minister had told him that lost European airline traffic could be made up for through a bilateral agreement with Pakistan. Which seems a bit optimistic to me, but what do I know.

Intercontinental flights are still likely to be more difficult, though. Since 2007, flights between Europe and the US have operated under a separate open skies agreement, and leaving the EU means we’re we’re about to fall out of that, too.  

Surely we’ll just revert to whatever rules there were before.

Apparently not. Airlines for America – a trade body for... well, you can probably guess that, too – has pointed out that, if we do, there are no historic rules to fall back on: there’s no aviation equivalent of the WTO.

The claim that flights are going to just stop is definitely a worst case scenario: in practice, we can probably negotiate a bunch of new agreements. But we’re already negotiating a lot of other things, and we’re on a deadline, so we’re tight for time.

In fact, we’re really tight for time. Airlines for America has also argued that – because so many tickets are sold a year or more in advance – airlines really need a new deal in place by March 2018, if they’re to have faith they can keep flying. So it’s asking for aviation to be prioritised in negotiations.

The only problem is, we can’t negotiate anything else until the EU decides we’ve made enough progress on the divorce bill and the rights of EU nationals. And the clock’s ticking.

This is just remoaning. Brexit will set us free.

A little bit, maybe. CAA’s Haines has also said he believes “talk of significant retrenchment is very much over-stated, and Brexit offers potential opportunities in other areas”. Falling out of Europe means falling out of European ownership rules, so itcould bring foreign capital into the UK aviation industry (assuming anyone still wants to invest, of course). It would also mean more flexibility on “slot rules”, by which airports have to hand out landing times, and which are I gather a source of some contention at the moment.

But Haines also pointed out that the UK has been one of the most influential contributors to European aviation regulations: leaving the European system will mean we lose that influence. And let’s not forget that it was European law that gave passengers the right to redress when things go wrong: if you’ve ever had a refund after long delays, you’ve got the EU to thank.

So: the planes may not stop flying. But the UK will have less influence over the future of aviation; passengers might have fewer consumer rights; and while it’s not clear that Brexit will mean vastly fewer flights, it’s hard to see how it will mean more, so between that and the slide in sterling, prices are likely to rise, too.

It’s not that Brexit is inevitably going to mean disaster. It’s just that it’ll take a lot of effort for very little obvious reward. Which is becoming something of a theme.

Still, we’ll be free of those bureaucrats at the ECJ, won’t be?

This’ll be a great comfort when we’re all holidaying in Grimsby.

Jonn Elledge edits the New Statesman's sister site CityMetric, and writes for the NS about subjects including politics, history and Brexit. You can find him on Twitter or Facebook.