Support 100 years of independent journalism.

  1. Science & Tech
20 July 2021

“It’s the poorest in society who are being surveilled”: the rise of citizen-scoring algorithms

A new report reveals that millions of people across the UK are being profiled by welfare-focused algorithms without their knowledge.

By Oscar Williams

In November 2018, the UN special rapporteur Philip Alston published a damning report on the UK’s poverty crisis. “We are witnessing the gradual disappearance of the postwar British welfare state behind a webpage and an algorithm,” Alston warned. “In its place, a digital welfare state is emerging. The impact on the human rights of the most vulnerable in the UK will be immense.”

Over the subsequent two and a half years, journalists and civil liberties activists have sought to uncover the extent to which algorithmic risk-profiling now underpins the welfare state. A new report, published today by the campaign group Big Brother Watch, reveals that dozens of councils and social housing providers are routinely profiling millions of citizens without their knowledge.

According to the report, which is based on hundreds of freedom of information requests, councils have now carried out fraud risk profiling on more than half a million housing benefit or council tax support applicants. Some 1.6 million social housing residents have been profiled by commercial algorithms to assess their likelihood of paying rent. And more than a quarter of a million people have been subject to data harvesting to identify their risk of being abused, or becoming homeless or unemployed.

“It’s the poorest in society who are being surveilled,” says Jake Hurfurt, head of investigations at Big Brother Watch. “If you look at all these systems, it’s social housing, it’s Universal Credit, it’s trying to target people and profile them for financial vulnerability or vulnerability to other things.” Hurfurt says that the campaign group is “very, very concerned” that the government is “creating essentially a surveillance state, where as a condition for state support, you have to undergo this kind of monitoring and profiling”.

By sowing distrust of welfare claimants and stripping councils of already limited resources, austerity has created the conditions for this technology to flourish. A patchwork of providers have sought to capitalise on councils’ need to cut costs. Technology produced by Xantura, a major provider, can reportedly save a local authority £127,000 over the course of a year.

Select and enter your email address Quick and essential guide to domestic and global politics from the New Statesman's politics team. A weekly newsletter helping you fit together the pieces of the global economic slowdown. The New Statesman’s global affairs newsletter, every Monday and Friday. The best of the New Statesman, delivered to your inbox every weekday morning. The New Statesman’s weekly environment email on the politics, business and culture of the climate and nature crises - in your inbox every Thursday. Our weekly culture newsletter – from books and art to pop culture and memes – sent every Friday. A weekly round-up of some of the best articles featured in the most recent issue of the New Statesman, sent each Saturday. A newsletter showcasing the finest writing from the ideas section and the NS archive, covering political ideas, philosophy, criticism and intellectual history - sent every Wednesday. Sign up to receive information regarding NS events, subscription offers & product updates.
  • Administration / Office
  • Arts and Culture
  • Board Member
  • Business / Corporate Services
  • Client / Customer Services
  • Communications
  • Construction, Works, Engineering
  • Education, Curriculum and Teaching
  • Environment, Conservation and NRM
  • Facility / Grounds Management and Maintenance
  • Finance Management
  • Health - Medical and Nursing Management
  • HR, Training and Organisational Development
  • Information and Communications Technology
  • Information Services, Statistics, Records, Archives
  • Infrastructure Management - Transport, Utilities
  • Legal Officers and Practitioners
  • Librarians and Library Management
  • Management
  • Marketing
  • OH&S, Risk Management
  • Operations Management
  • Planning, Policy, Strategy
  • Printing, Design, Publishing, Web
  • Projects, Programs and Advisors
  • Property, Assets and Fleet Management
  • Public Relations and Media
  • Purchasing and Procurement
  • Quality Management
  • Science and Technical Research and Development
  • Security and Law Enforcement
  • Service Delivery
  • Sport and Recreation
  • Travel, Accommodation, Tourism
  • Wellbeing, Community / Social Services
I consent to New Statesman Media Group collecting my details provided via this form in accordance with the Privacy Policy
THANK YOU

[See also: How citizen-scoring algorithms are being used by local government in the UK​]

Content from our partners
The cost-of-living crisis is hitting small businesses – Liz Truss must act
How industry is key for net zero
How to ensure net zero brings good growth and green jobs

In recent years a number of councils have adopted Xantura’s OneView product. The predictive analytics software draws upon a range of structured and unstructured data to assess a household’s risk of debt or homelessness, as well as harm related to Covid-19. “A proposal document from Shropshire Council suggested that even details of sexual habits and anger management could be gleaned in case notes,” Big Brother Watch found. 

But while the tool harvests huge amounts of data, those who are subject to its analysis are unlikely to know about it. A data protection assessment carried out by Thurrock Council revealed it hadn’t sought individuals’ consent before analysing the data for this purpose.

Hurfurt says it was difficult to find individuals who knew they had been profiled by local authorities’ risk-scoring algorithms. But one woman, who had received housing support, told Big Brother Watch she was “disgusted” to have learnt that her data had been used in this way.

“I’ve noticed the amount of evidence I’ve been asked for has changed over the years, which makes it really stressful. I’ve been made to go through all my bank statements line by line with an assessor, which made me feel like a criminal. Now I wonder if it’s because a machine decided, for reasons unknown, I could be a fraudster. It feels very unjust for people like me in genuine need, to know I’m being scrutinised and not believed over evidence I provide.”

Even local authorities appear to have an incomplete understanding of how the technology works, having passed some of Big Brother Watch’s questions to providers. Hurfurt says it is “very much happening in the shadows and this is something we’re concerned about. You often only know you’ve been affected by this when it’s too late. The councils need to be much more transparent about what they’re doing with people’s data.”

[See also: How Universal Credit is one of the least generous welfare systems in Europe]

Over the last four years, the number of councils using predictive analytics to assess benefits claimants’ fraud risk has dropped from around 80 to 50, according to the research. One council said the technology had “yet to realise most of its predicted financial efficiencies”; the introduction of Universal Credit has also made the technology less financially appealing. But Hurfurt believes that another factor is at play too: a decision by the government to centralise the analysis. 

“The Department for Work and Pensions is running centralised housing benefit fraud risk-scoring,” Hurfurt said. “So, the risk-based verification stuff is when you apply for benefits; the DWP is now risk-scoring everyone who receives housing benefit, and forcing councils to do full case reviews on the riskiest 400,000 people.”

Despite the drop in fraud risk-scoring at a local level, Hurfurt says that the rising popularity of other forms of predictive analytics, such as assessing individuals’ risk of harm, means that the number of councils using some kind of algorithmic profiling has remained at the same level.

Big Brother Watch is now calling for the Information Commissioner’s Office to introduce a public register of algorithms used in public sector decision-making. “[Councils] wade through all sorts of quite dodgy justifications, whereas if you have to be public about it and have a register, at least, that would be open to question and open to challenge,” says Hurfurt. “At the moment nobody really knows what’s going on.”

[See also: Poverty by algorithm: The Universal Credit design flaw that leaves people penniless]