The DWP's bogus psychometric tests are dystopian, but randomised trials aren't the problem

The solution is more trials, not fewer.

The DWP is under fire for mandating benefit claimants to carry out bogus psychometric tests. The Guardian's Shiv Malik reports:

The test called My Strengths, devised by Downing Street's behavioural insights or "nudge" unit, has been exposed by bloggers as a sham with results having no relation to the answers given.

Some of the 48 statements on the DWP test include: "I never go out of my way to visit museums," and: "I have not created anything of beauty in the last year." People are asked to grade their answers from "very much like me" to "very much unlike me".

When those being tested complete the official online questionnaire, they are assigned a set of five positive "strengths" including "love of learning" and "curiosity" and "originality".

However, those taking the supposed psychological survey have found that by clicking on the same answer repeatedly, users will get the same set of personality results as those entering a completely opposite set of answers.

The aim behind the "test" is apparently to prevent the claimants with motivating statements about themselves, in an effort to encourage them to find work. The behavioural insight team itself describes the process as:

Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

But it's worth being clear about why "My Strengths" is so unpleasant. The problem with it is that unemployed people are being lied to in an effort to psychologically manipulate them into a mental state where they will do what the Government want. That sounds like a sentence from a dystopian future, but it's a fairly straight description of what is happening.

But Steve Walker, the blogger who broke the story, goes further, attacking the test because it's part of a government trial.

Walker writes that the trial breaches principals established by the EU's body for co-ordinating R&D which state that the medical rule of informed consent "remains valid for any other kind of research".

Principles aren't laws, so there's not a legal case to answer here; but even if we're just arguing morality, establishing medical-grade requirements of informed consent in social research would be a terrible idea.

The rise of randomised controlled trials (and, to a lesser extent, randomised trials and controlled trials – not everything can be both randomised and controlled) in social research is one of the most promising trends of the last decade. It's good not only because it can increase our knowledge of how to fight social problems in employment, education and crime, but because it's barely different from what's being done anyway.

Most providers of the government's work programme – which involves more than just the unpaid mandatory labour that has made it infamous –  have neither the staff, time or money to offer every service to every person who comes through their doors. When they're trying to decide who gets to go on the CV workshops and who doesn't, the obvious thing to do is to chart which groups improve the most when given the workshops. This is doubly true if you don't actually know whether CV workshops help or harm; keeping a good eye on the results is invaluable.

When it comes to trials, the biggest crime of the BIT isn't that they did them, but that they didn't do them enough. As a report from December shows, the team aggregated together three completely different changes:

  1. Making sure every customer talks about getting back to work on their first day (not after 2 weeks) by cutting down and reorganising processes;
  2. Introducing stretching commitment devices which focus on what the job seeker will do for the whole of the next fortnight. This replaces the present system where advisors ask if job seekers have done three job search activities in each of the previous two weeks;
  3. Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

Overall, apparently, "job seekers in the treatment group are 15-20 per cent more likely than those in the control group to be off benefits 13 weeks after signing on" – but the trial is so badly designed that we can't see which of these treatments helped the most. It could well be the case that the My Strengths test actually hindered jobseekers, but the effect was hidden by the benefits of "cutting down and reorganising processes".

The government needs to behave ethically when dealing with its citizens, and that goes whether or not it is done in the framework of a formal trial. But if it does try a new ethical way of helping jobseekers, the last thing we want to do is discourage it from measuring the results.

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Getty
Show Hide image

Wrists, knees, terrible rages – I felt overwhelmed when Barry came to see me

I teach my registrars to be aware how a consultation is making them feel: that can give valuable clues to the patient’s own emotional state.

To begin with, it seemed that Barry’s wrists were the problem. He told me about the pain he was experiencing, the pins and needles that came and went in his hands. I started to examine him. His palms were calloused, his fingers thick and stubby, veterans of the heavy work he’d undertaken throughout his 57 years. Even as I assessed this first problem, he mentioned his knees. I moved on to look at those. Then it was his back. I couldn’t get to grips with one thing before he veered to the next.

I teach my registrars to be aware how a consultation is making them feel: that can give valuable clues to the patient’s own emotional state. Barry was making me feel overwhelmed, the more so as I learned that he’d been experiencing all these problems for years.

“Why are you coming to see me about them now,” I asked, “rather than six months ago – or in six months’ time?”

“I need some time off, doc.”

There was something about the way he wouldn’t meet my gaze. And again, that feeling of being overwhelmed.

“What’s going on at work?” I asked him.

His tone hardened as he told me how he’d lost his temper a couple of days earlier. How one of the others had been winding him up, and something inside him had snapped, and he’d taken a swing at his workmate and landed a punch.

Barry had walked out and hadn’t been back. I tried to find out if he’d heard from his boss about the incident, if he knew what was likely to happen next.

He told me he didn’t care.

We talked some more. I learned that he’d been uncharacteristically short-tempered for months; his partner was fed up with being shouted at. Sleep had gone to pot, and Barry had taken to drinking heavily to knock himself out at night. He was smoking twice his usual amount. Men like Barry often don’t experience depression as classic low mood and tearfulness; they become filled with rage and turn in on themselves, repelling those closest to them in the process.

Depression is a complex condition, with roots that can frequently be traced right back to childhood experiences, but bouts are often precipitated by problems with relationships, work, money, or health. In Barry’s case, the main factor turned out to be his job. He’d been an HGV driver but at the start of the year his company had lost its operator’s licence. To keep the business afloat, his boss had diversified. Barry hated what he now had to do. He was now a “catcher”.

I didn’t know what that meant. Getting up at the crack of dawn, he told me, driving to some factory farm somewhere, entering huge sheds and spending hours catching chickens, thousands upon thousands of them, shoving them into crates, stashing the crates on a lorry, working under relentless pressure to get the sheds cleared and the birds off to the next stage of the food production chain.

“It’s a young man’s game,” he told me. “It’s crippling me, all that bending and catching.”

It wasn’t really his joints, though. Men like Barry can find it hard to talk about difficult emotion, but it was there in his eyes. I had a sudden understanding: Barry, capturing bird after panicking bird, stuffing them into the transport containers, the air full of alarmed clucking and dislodged feathers. Hour after hour of it. It was traumatising him, but he couldn’t admit anything so poncey.

“I just want to get back to driving.”

That would mean landing a new job, and he doubted he would be able to do so, not at his age. He couldn’t take just any old work, either: he had to earn a decent wage to keep up with a still sizeable mortgage.

We talked about how antidepressants might improve his symptoms, and made a plan to tackle the alcohol. I signed him off to give him some respite and a chance to look for new work – the one thing that was going to resolve his depression. But in the meantime, he felt as trapped as the chickens that he cornered, day after soul-destroying day.

Phil Whitaker’s novel “Sister Sebastian’s Library” will be published by Salt in September

This article first appeared in the 21 July 2016 issue of the New Statesman, The English Revolt