The DWP's bogus psychometric tests are dystopian, but randomised trials aren't the problem

The solution is more trials, not fewer.

The DWP is under fire for mandating benefit claimants to carry out bogus psychometric tests. The Guardian's Shiv Malik reports:

The test called My Strengths, devised by Downing Street's behavioural insights or "nudge" unit, has been exposed by bloggers as a sham with results having no relation to the answers given.

Some of the 48 statements on the DWP test include: "I never go out of my way to visit museums," and: "I have not created anything of beauty in the last year." People are asked to grade their answers from "very much like me" to "very much unlike me".

When those being tested complete the official online questionnaire, they are assigned a set of five positive "strengths" including "love of learning" and "curiosity" and "originality".

However, those taking the supposed psychological survey have found that by clicking on the same answer repeatedly, users will get the same set of personality results as those entering a completely opposite set of answers.

The aim behind the "test" is apparently to prevent the claimants with motivating statements about themselves, in an effort to encourage them to find work. The behavioural insight team itself describes the process as:

Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

But it's worth being clear about why "My Strengths" is so unpleasant. The problem with it is that unemployed people are being lied to in an effort to psychologically manipulate them into a mental state where they will do what the Government want. That sounds like a sentence from a dystopian future, but it's a fairly straight description of what is happening.

But Steve Walker, the blogger who broke the story, goes further, attacking the test because it's part of a government trial.

Walker writes that the trial breaches principals established by the EU's body for co-ordinating R&D which state that the medical rule of informed consent "remains valid for any other kind of research".

Principles aren't laws, so there's not a legal case to answer here; but even if we're just arguing morality, establishing medical-grade requirements of informed consent in social research would be a terrible idea.

The rise of randomised controlled trials (and, to a lesser extent, randomised trials and controlled trials – not everything can be both randomised and controlled) in social research is one of the most promising trends of the last decade. It's good not only because it can increase our knowledge of how to fight social problems in employment, education and crime, but because it's barely different from what's being done anyway.

Most providers of the government's work programme – which involves more than just the unpaid mandatory labour that has made it infamous –  have neither the staff, time or money to offer every service to every person who comes through their doors. When they're trying to decide who gets to go on the CV workshops and who doesn't, the obvious thing to do is to chart which groups improve the most when given the workshops. This is doubly true if you don't actually know whether CV workshops help or harm; keeping a good eye on the results is invaluable.

When it comes to trials, the biggest crime of the BIT isn't that they did them, but that they didn't do them enough. As a report from December shows, the team aggregated together three completely different changes:

  1. Making sure every customer talks about getting back to work on their first day (not after 2 weeks) by cutting down and reorganising processes;
  2. Introducing stretching commitment devices which focus on what the job seeker will do for the whole of the next fortnight. This replaces the present system where advisors ask if job seekers have done three job search activities in each of the previous two weeks;
  3. Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

Overall, apparently, "job seekers in the treatment group are 15-20 per cent more likely than those in the control group to be off benefits 13 weeks after signing on" – but the trial is so badly designed that we can't see which of these treatments helped the most. It could well be the case that the My Strengths test actually hindered jobseekers, but the effect was hidden by the benefits of "cutting down and reorganising processes".

The government needs to behave ethically when dealing with its citizens, and that goes whether or not it is done in the framework of a formal trial. But if it does try a new ethical way of helping jobseekers, the last thing we want to do is discourage it from measuring the results.

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Getty
Show Hide image

A swimming pool and a bleeding toe put my medical competency in doubt

Doctors are used to contending with Google. Sometimes the search engine wins. 

The brutal heatwave affecting southern Europe this summer has become known among locals as “Lucifer”. Having just returned from Italy, I fully understand the nickname. An early excursion caused the beginnings of sunstroke, so we abandoned plans to explore the cultural heritage of the Amalfi region and strayed no further than five metres from the hotel pool for the rest of the week.

The children were delighted, particularly my 12-year-old stepdaughter, Gracie, who proceeded to spend hours at a time playing in the water. Towelling herself after one long session, she noticed something odd.

“What’s happened there?” she asked, holding her foot aloft in front of my face.

I inspected the proffered appendage: on the underside of her big toe was an oblong area of glistening red flesh that looked like a chunk of raw steak.

“Did you injure it?”

She shook her head. “It doesn’t hurt at all.”

I shrugged and said she must have grazed it. She wasn’t convinced, pointing out that she would remember if she had done that. She has great faith in plasters, though, and once it was dressed she forgot all about it. I dismissed it, too, assuming it was one of those things.

By the end of the next day, the pulp on the underside of all of her toes looked the same. As the doctor in the family, I felt under some pressure to come up with an explanation. I made up something about burns from the hot paving slabs around the pool. Gracie didn’t say as much, but her look suggested a dawning scepticism over my claims to hold a medical degree.

The next day, Gracie and her new-found holiday playmate, Eve, abruptly terminated a marathon piggy-in-the-middle session in the pool with Eve’s dad. “Our feet are bleeding,” they announced, somewhat incredulously. Sure enough, bright-red blood was flowing, apparently painlessly, from the bottoms of their big toes.

Doctors are used to contending with Google. Often, what patients discover on the internet causes them undue alarm, and our role is to provide context and reassurance. But not infrequently, people come across information that outstrips our knowledge. On my return from our room with fresh supplies of plasters, my wife looked up from her sun lounger with an air of quiet amusement.

“It’s called ‘pool toe’,” she said, handing me her iPhone. The page she had tracked down described the girls’ situation exactly: friction burns, most commonly seen in children, caused by repetitive hopping about on the abrasive floors of swimming pools. Doctors practising in hot countries must see it all the time. I doubt it presents often to British GPs.

I remained puzzled about the lack of pain. The injuries looked bad, but neither Gracie nor Eve was particularly bothered. Here the internet drew a blank, but I suspect it has to do with the “pruning” of our skin that we’re all familiar with after a soak in the bath. This only occurs over the pulps of our fingers and toes. It was once thought to be caused by water diffusing into skin cells, making them swell, but the truth is far more fascinating.

The wrinkling is an active process, triggered by immersion, in which the blood supply to the pulp regions is switched off, causing the skin there to shrink and pucker. This creates the biological equivalent of tyre treads on our fingers and toes and markedly improves our grip – of great evolutionary advantage when grasping slippery fish in a river, or if trying to maintain balance on slick wet rocks.

The flip side of this is much greater friction, leading to abrasion of the skin through repeated micro-trauma. And the lack of blood flow causes nerves to shut down, depriving us of the pain that would otherwise alert us to the ongoing tissue damage. An adaptation that helped our ancestors hunt in rivers proves considerably less use on a modern summer holiday.

I may not have seen much of the local heritage, but the trip to Italy taught me something new all the same. 

This article first appeared in the 17 August 2017 issue of the New Statesman, Trump goes nuclear