The DWP's bogus psychometric tests are dystopian, but randomised trials aren't the problem

The solution is more trials, not fewer.

The DWP is under fire for mandating benefit claimants to carry out bogus psychometric tests. The Guardian's Shiv Malik reports:

The test called My Strengths, devised by Downing Street's behavioural insights or "nudge" unit, has been exposed by bloggers as a sham with results having no relation to the answers given.

Some of the 48 statements on the DWP test include: "I never go out of my way to visit museums," and: "I have not created anything of beauty in the last year." People are asked to grade their answers from "very much like me" to "very much unlike me".

When those being tested complete the official online questionnaire, they are assigned a set of five positive "strengths" including "love of learning" and "curiosity" and "originality".

However, those taking the supposed psychological survey have found that by clicking on the same answer repeatedly, users will get the same set of personality results as those entering a completely opposite set of answers.

The aim behind the "test" is apparently to prevent the claimants with motivating statements about themselves, in an effort to encourage them to find work. The behavioural insight team itself describes the process as:

Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

But it's worth being clear about why "My Strengths" is so unpleasant. The problem with it is that unemployed people are being lied to in an effort to psychologically manipulate them into a mental state where they will do what the Government want. That sounds like a sentence from a dystopian future, but it's a fairly straight description of what is happening.

But Steve Walker, the blogger who broke the story, goes further, attacking the test because it's part of a government trial.

Walker writes that the trial breaches principals established by the EU's body for co-ordinating R&D which state that the medical rule of informed consent "remains valid for any other kind of research".

Principles aren't laws, so there's not a legal case to answer here; but even if we're just arguing morality, establishing medical-grade requirements of informed consent in social research would be a terrible idea.

The rise of randomised controlled trials (and, to a lesser extent, randomised trials and controlled trials – not everything can be both randomised and controlled) in social research is one of the most promising trends of the last decade. It's good not only because it can increase our knowledge of how to fight social problems in employment, education and crime, but because it's barely different from what's being done anyway.

Most providers of the government's work programme – which involves more than just the unpaid mandatory labour that has made it infamous –  have neither the staff, time or money to offer every service to every person who comes through their doors. When they're trying to decide who gets to go on the CV workshops and who doesn't, the obvious thing to do is to chart which groups improve the most when given the workshops. This is doubly true if you don't actually know whether CV workshops help or harm; keeping a good eye on the results is invaluable.

When it comes to trials, the biggest crime of the BIT isn't that they did them, but that they didn't do them enough. As a report from December shows, the team aggregated together three completely different changes:

  1. Making sure every customer talks about getting back to work on their first day (not after 2 weeks) by cutting down and reorganising processes;
  2. Introducing stretching commitment devices which focus on what the job seeker will do for the whole of the next fortnight. This replaces the present system where advisors ask if job seekers have done three job search activities in each of the previous two weeks;
  3. Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

Overall, apparently, "job seekers in the treatment group are 15-20 per cent more likely than those in the control group to be off benefits 13 weeks after signing on" – but the trial is so badly designed that we can't see which of these treatments helped the most. It could well be the case that the My Strengths test actually hindered jobseekers, but the effect was hidden by the benefits of "cutting down and reorganising processes".

The government needs to behave ethically when dealing with its citizens, and that goes whether or not it is done in the framework of a formal trial. But if it does try a new ethical way of helping jobseekers, the last thing we want to do is discourage it from measuring the results.

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Photo: Getty Images
Show Hide image

There are risks as well as opportunities ahead for George Osborne

The Chancellor is in a tight spot, but expect his political wiles to be on full display, says Spencer Thompson.

The most significant fiscal event of this parliament will take place in late November, when the Chancellor presents the spending review setting out his plans for funding government departments over the next four years. This week, across Whitehall and up and down the country, ministers, lobbyists, advocacy groups and town halls are busily finalising their pitches ahead of Friday’s deadline for submissions to the review

It is difficult to overstate the challenge faced by the Chancellor. Under his current spending forecast and planned protections for the NHS, schools, defence and international aid spending, other areas of government will need to be cut by 16.4 per cent in real terms between 2015/16 and 2019/20. Focusing on services spending outside of protected areas, the cumulative cut will reach 26.5 per cent. Despite this, the Chancellor nonetheless has significant room for manoeuvre.

Firstly, under plans unveiled at the budget, the government intends to expand capital investment significantly in both 2018-19 and 2019-20. Over the last parliament capital spending was cut by around a quarter, but between now and 2019-20 it will grow by almost 20 per cent. How this growth in spending should be distributed across departments and between investment projects should be at the heart of the spending review.

In a paper published on Monday, we highlighted three urgent priorities for any additional capital spending: re-balancing transport investment away from London and the greater South East towards the North of England, a £2bn per year boost in public spending on housebuilding, and £1bn of extra investment per year in energy efficiency improvements for fuel-poor households.

Secondly, despite the tough fiscal environment, the Chancellor has the scope to fund a range of areas of policy in dire need of extra resources. These include social care, where rising costs at a time of falling resources are set to generate a severe funding squeeze for local government, 16-19 education, where many 6th-form and FE colleges are at risk of great financial difficulty, and funding a guaranteed paid job for young people in long-term unemployment. Our paper suggests a range of options for how to put these and other areas of policy on a sustainable funding footing.

There is a political angle to this as well. The Conservatives are keen to be seen as a party representing all working people, as shown by the "blue-collar Conservatism" agenda. In addition, the spending review offers the Conservative party the opportunity to return to ‘Compassionate Conservatism’ as a going concern.  If they are truly serious about being seen in this light, this should be reflected in a social investment agenda pursued through the spending review that promotes employment and secures a future for public services outside the NHS and schools.

This will come at a cost, however. In our paper, we show how the Chancellor could fund our package of proposed policies without increasing the pain on other areas of government, while remaining consistent with the government’s fiscal rules that require him to reach a surplus on overall government borrowing by 2019-20. We do not agree that the Government needs to reach a surplus in that year. But given this target wont be scrapped ahead of the spending review, we suggest that he should target a slightly lower surplus in 2019/20 of £7bn, with the deficit the year before being £2bn higher. In addition, we propose several revenue-raising measures in line with recent government tax policy that together would unlock an additional £5bn of resource for government departments.

Make no mistake, this will be a tough settlement for government departments and for public services. But the Chancellor does have a range of options open as he plans the upcoming spending review. Expect his reputation as a highly political Chancellor to be on full display.

Spencer Thompson is economic analyst at IPPR