Support 100 years of independent journalism.

  1. Spotlight
  2. Devolution
17 August 2020

Westminster’s U-turn over exam grades is a case study in how not to use algorithms

With no actual data, the government fell foul of the “garbage in, garbage out” problem. 

By Stephen Bush

On A-levels and GCSEs, the Westminster government has followed in the footsteps of the Scottish, Welsh and Northern Irish administrations. It has abandoned its attempt to allocate exam results to students in England calculated using modelling of how they would have done, based on the performance of previous year groups.

The row is a case study in how not to use algorithms in public policy: because an algorithm is a process, not an event.

If you were a PR company, you could make an algorithm to judge how well-disposed I would be to your email, by throwing, say, what I had for lunch and the Arsenal result into a spreadsheet. But if you don’t know what I had for lunch or what the Arsenal score was, you couldn’t gauge my mood using historical variables on what the Arsenal score had been, on average, that week over the past five years, no matter how hard you fiddled with the data.

That’s the problem being faced by the exams watchdogs in England, Scotland and Northern Ireland. A pupil taking their GCSEs in England and Northern Ireland or their Nationals in Scotland in 2020 has not sat a centralised examination since 2015. We know very little about these pupils and cannot reasonably gauge what their GCSE results are going to be with any degree of accuracy. And because exams for Highers and A-levels are considerably tougher than those for GCSEs, we can’t use those to help us reliably allocate A-level passes either.

An algorithm is a way you process information – not something you can use to create new information from thin air. If you have no actual data, you have what bankers call the “garbage in, garbage out” problem: if you have rubbish or non-existent data, it doesn’t matter how sophisticated the spreadsheet you run it through is – you can’t make the resulting numbers make sense.

Select and enter your email address Quick and essential guide to domestic and global politics from the New Statesman's politics team. A weekly newsletter helping you fit together the pieces of the global economic slowdown. The New Statesman’s global affairs newsletter, every Monday and Friday. The best of the New Statesman, delivered to your inbox every weekday morning. The New Statesman’s weekly environment email on the politics, business and culture of the climate and nature crises - in your inbox every Thursday. Our weekly culture newsletter – from books and art to pop culture and memes – sent every Friday. A weekly round-up of some of the best articles featured in the most recent issue of the New Statesman, sent each Saturday. A newsletter showcasing the finest writing from the ideas section and the NS archive, covering political ideas, philosophy, criticism and intellectual history - sent every Wednesday. Sign up to receive information regarding NS events, subscription offers & product updates.
  • Administration / Office
  • Arts and Culture
  • Board Member
  • Business / Corporate Services
  • Client / Customer Services
  • Communications
  • Construction, Works, Engineering
  • Education, Curriculum and Teaching
  • Environment, Conservation and NRM
  • Facility / Grounds Management and Maintenance
  • Finance Management
  • Health - Medical and Nursing Management
  • HR, Training and Organisational Development
  • Information and Communications Technology
  • Information Services, Statistics, Records, Archives
  • Infrastructure Management - Transport, Utilities
  • Legal Officers and Practitioners
  • Librarians and Library Management
  • Management
  • Marketing
  • OH&S, Risk Management
  • Operations Management
  • Planning, Policy, Strategy
  • Printing, Design, Publishing, Web
  • Projects, Programs and Advisors
  • Property, Assets and Fleet Management
  • Public Relations and Media
  • Purchasing and Procurement
  • Quality Management
  • Science and Technical Research and Development
  • Security and Law Enforcement
  • Service Delivery
  • Sport and Recreation
  • Travel, Accommodation, Tourism
  • Wellbeing, Community / Social Services
I consent to New Statesman Media Group collecting my details provided via this form in accordance with the Privacy Policy

Because in Wales, the education system still uses modular assessment, the government was able to produce a more popular – or perhaps, more accurately, less unpopular – model of doing things, because GCSE coursework and AS-level results meant that pupils had at least a partial hand in how their results were generated. But they have had to abandon their model, too, because persisting with it would have meant giving Welsh pupils considerably worse grades than children elsewhere in the United Kingdom.

Ultimately an algorithm is not a solution to having insufficient data. You can’t generate exam results without exams: it really is that simple.