The DWP's bogus psychometric tests are dystopian, but randomised trials aren't the problem

The solution is more trials, not fewer.

The DWP is under fire for mandating benefit claimants to carry out bogus psychometric tests. The Guardian's Shiv Malik reports:

The test called My Strengths, devised by Downing Street's behavioural insights or "nudge" unit, has been exposed by bloggers as a sham with results having no relation to the answers given.

Some of the 48 statements on the DWP test include: "I never go out of my way to visit museums," and: "I have not created anything of beauty in the last year." People are asked to grade their answers from "very much like me" to "very much unlike me".

When those being tested complete the official online questionnaire, they are assigned a set of five positive "strengths" including "love of learning" and "curiosity" and "originality".

However, those taking the supposed psychological survey have found that by clicking on the same answer repeatedly, users will get the same set of personality results as those entering a completely opposite set of answers.

The aim behind the "test" is apparently to prevent the claimants with motivating statements about themselves, in an effort to encourage them to find work. The behavioural insight team itself describes the process as:

Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

But it's worth being clear about why "My Strengths" is so unpleasant. The problem with it is that unemployed people are being lied to in an effort to psychologically manipulate them into a mental state where they will do what the Government want. That sounds like a sentence from a dystopian future, but it's a fairly straight description of what is happening.

But Steve Walker, the blogger who broke the story, goes further, attacking the test because it's part of a government trial.

Walker writes that the trial breaches principals established by the EU's body for co-ordinating R&D which state that the medical rule of informed consent "remains valid for any other kind of research".

Principles aren't laws, so there's not a legal case to answer here; but even if we're just arguing morality, establishing medical-grade requirements of informed consent in social research would be a terrible idea.

The rise of randomised controlled trials (and, to a lesser extent, randomised trials and controlled trials – not everything can be both randomised and controlled) in social research is one of the most promising trends of the last decade. It's good not only because it can increase our knowledge of how to fight social problems in employment, education and crime, but because it's barely different from what's being done anyway.

Most providers of the government's work programme – which involves more than just the unpaid mandatory labour that has made it infamous –  have neither the staff, time or money to offer every service to every person who comes through their doors. When they're trying to decide who gets to go on the CV workshops and who doesn't, the obvious thing to do is to chart which groups improve the most when given the workshops. This is doubly true if you don't actually know whether CV workshops help or harm; keeping a good eye on the results is invaluable.

When it comes to trials, the biggest crime of the BIT isn't that they did them, but that they didn't do them enough. As a report from December shows, the team aggregated together three completely different changes:

  1. Making sure every customer talks about getting back to work on their first day (not after 2 weeks) by cutting down and reorganising processes;
  2. Introducing stretching commitment devices which focus on what the job seeker will do for the whole of the next fortnight. This replaces the present system where advisors ask if job seekers have done three job search activities in each of the previous two weeks;
  3. Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

Overall, apparently, "job seekers in the treatment group are 15-20 per cent more likely than those in the control group to be off benefits 13 weeks after signing on" – but the trial is so badly designed that we can't see which of these treatments helped the most. It could well be the case that the My Strengths test actually hindered jobseekers, but the effect was hidden by the benefits of "cutting down and reorganising processes".

The government needs to behave ethically when dealing with its citizens, and that goes whether or not it is done in the framework of a formal trial. But if it does try a new ethical way of helping jobseekers, the last thing we want to do is discourage it from measuring the results.

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Show Hide image

Does the UK care enough about climate change to admit it is part of the problem?

The government’s energy policies make can make it hard to decipher its commitment to emissions reduction.

“People tell me it’s ridiculous to be flying for a climate change project but you have to get real with it, I mean I can’t cycle across the Southern ocean,” says Daniel Price, an environmental scientist from London. As founder of Pole-to-Paris, Price is about to complete a 17,000km bike ride from the Antarctic to the Arc de Triomphe.

Price came up with the idea in an effort to raise public awareness of COP21, the UN Climate Change Conference taking place in Paris next week. During the trip he’s faced a succession of set-backs: from the discovery that boats were prohibitively expensive, to diplomatic tensions scuppering his Russian visa plans. Yet the darkest moments were when he became overwhelmed by the magnitude of his own mission. “There were difficult times when I just thought, ‘What is the point of this’?” he says. “Cycling round the world is nowhere near enough to engage people.” 

As world leaders descend on Paris, many questions remain unanswered. Not least how much support developing nations will receive in tackling the effects of climate change. New research commissioned by Oxfam claims that such costs could rise to £1.7tn a year by 2050. But with cuts kicking in at home, the need to deliver “climate justice” abroad feels like a bigger ask than ever.

So does Britain really care enough about climate change to accept its full part in this burden? The government’s energy policies make can make it hard to decipher its commitment to emissions reduction. In September, however, it did pledge £5.8bn from the foreign aid fund to helping poorer nations combat climate change (twice that promised by China and the United States). And there’s evidence to suggest that we, as a public, may also care more than we think.

In America attitudes are much darker; in the dismissive words of Donald Trump “It’s called the weather”. Not least since, as a recent study proves, over the last twenty years corporations have systematically spread scepticism about the science. “The contrarian efforts have been so effective," says the author Justin Farrell, a Yale sociologist, "that they have made it difficult for ordinary Americans to even know who to trust.” 

And what about in China, the earth's biggest polluter? Single-party rule and the resulting lack of public discussion would seem to be favouring action on the environment. The government has recently promised to reach "peak" emissions by 2030, to quadruple solar installations, and to commit $3.1bn to help low-income countries adapt to the changing world. Christiana Figueres, the UN’s chief climate official, has even lauded the country for taking “undisputed leadership” on climate change mitigation.

Yet this surge of policy could mask the most troubling reality of all: that, when it comes to climate change, the Chinese are the least concerned citizenship in the world. Only 18 per cent of Chinese see the issue as a very serious problem, down 23 percentage points from five years ago, and 36 points behind the global median.

A new study by political economist Dr Alex Lo has concluded that the country’s reduced political debate could be to blame for the lack of concern. “In China popular environmentalism is biased towards immediate environmental threats”, such as desertification and pollution, Lo writes, “giving little impetus to a morally driven climate change movement”.

For the international community, all is well and good as long as the Chinese government continues along its current trajectory. But without an engaged public to hold it to account there’s always a chance its promises may fade into thin air.

So perhaps the UK’s tendency to moan about how hard it is to care about the (seemingly) remote impacts of climate change isn’t all bad. At least we know it is something worth moaning about. And perhaps we care more than we let on to each other.

Statistics published this summer by the Department of Energy and Climate Change reveal that three quarters of the British public support subsidies for renewable energy, despite only 10 per cent thinking that the figure is that high. “Even if the public think the consensus is not there, there are encouraging signs that it is,” says Liz Callegari, Head of Campaigns at WWF. “Concern for climate change is growing.”

As Price puts it, “You can think of climate change as this kind of marathon effort that we have to address and in Paris we just have to get people walking across the start line together”. Maybe then we will all be ready to run.

India Bourke is the New Statesman's editorial assistant.