The DWP's bogus psychometric tests are dystopian, but randomised trials aren't the problem

The solution is more trials, not fewer.

The DWP is under fire for mandating benefit claimants to carry out bogus psychometric tests. The Guardian's Shiv Malik reports:

The test called My Strengths, devised by Downing Street's behavioural insights or "nudge" unit, has been exposed by bloggers as a sham with results having no relation to the answers given.

Some of the 48 statements on the DWP test include: "I never go out of my way to visit museums," and: "I have not created anything of beauty in the last year." People are asked to grade their answers from "very much like me" to "very much unlike me".

When those being tested complete the official online questionnaire, they are assigned a set of five positive "strengths" including "love of learning" and "curiosity" and "originality".

However, those taking the supposed psychological survey have found that by clicking on the same answer repeatedly, users will get the same set of personality results as those entering a completely opposite set of answers.

The aim behind the "test" is apparently to prevent the claimants with motivating statements about themselves, in an effort to encourage them to find work. The behavioural insight team itself describes the process as:

Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

But it's worth being clear about why "My Strengths" is so unpleasant. The problem with it is that unemployed people are being lied to in an effort to psychologically manipulate them into a mental state where they will do what the Government want. That sounds like a sentence from a dystopian future, but it's a fairly straight description of what is happening.

But Steve Walker, the blogger who broke the story, goes further, attacking the test because it's part of a government trial.

Walker writes that the trial breaches principals established by the EU's body for co-ordinating R&D which state that the medical rule of informed consent "remains valid for any other kind of research".

Principles aren't laws, so there's not a legal case to answer here; but even if we're just arguing morality, establishing medical-grade requirements of informed consent in social research would be a terrible idea.

The rise of randomised controlled trials (and, to a lesser extent, randomised trials and controlled trials – not everything can be both randomised and controlled) in social research is one of the most promising trends of the last decade. It's good not only because it can increase our knowledge of how to fight social problems in employment, education and crime, but because it's barely different from what's being done anyway.

Most providers of the government's work programme – which involves more than just the unpaid mandatory labour that has made it infamous –  have neither the staff, time or money to offer every service to every person who comes through their doors. When they're trying to decide who gets to go on the CV workshops and who doesn't, the obvious thing to do is to chart which groups improve the most when given the workshops. This is doubly true if you don't actually know whether CV workshops help or harm; keeping a good eye on the results is invaluable.

When it comes to trials, the biggest crime of the BIT isn't that they did them, but that they didn't do them enough. As a report from December shows, the team aggregated together three completely different changes:

  1. Making sure every customer talks about getting back to work on their first day (not after 2 weeks) by cutting down and reorganising processes;
  2. Introducing stretching commitment devices which focus on what the job seeker will do for the whole of the next fortnight. This replaces the present system where advisors ask if job seekers have done three job search activities in each of the previous two weeks;
  3. Building psychological resilience and wellbeing for those who are still claiming after 8 weeks through ‘expressive writing’ and strengths identification.

Overall, apparently, "job seekers in the treatment group are 15-20 per cent more likely than those in the control group to be off benefits 13 weeks after signing on" – but the trial is so badly designed that we can't see which of these treatments helped the most. It could well be the case that the My Strengths test actually hindered jobseekers, but the effect was hidden by the benefits of "cutting down and reorganising processes".

The government needs to behave ethically when dealing with its citizens, and that goes whether or not it is done in the framework of a formal trial. But if it does try a new ethical way of helping jobseekers, the last thing we want to do is discourage it from measuring the results.

Alex Hern is a technology reporter for the Guardian. He was formerly staff writer at the New Statesman. You should follow Alex on Twitter.

Getty
Show Hide image

I was wrong about Help to Buy - but I'm still glad it's gone

As a mortgage journalist in 2013, I was deeply sceptical of the guarantee scheme. 

If you just read the headlines about Help to Buy, you could be under the impression that Theresa May has just axed an important scheme for first-time buyers. If you're on the left, you might conclude that she is on a mission to make life worse for ordinary working people. If you just enjoy blue-on-blue action, it's a swipe at the Chancellor she sacked, George Osborne.

Except it's none of those things. Help to Buy mortgage guarantee scheme is a policy that actually worked pretty well - despite the concerns of financial journalists including me - and has served its purpose.

When Osborne first announced Help to Buy in 2013, it was controversial. Mortgage journalists, such as I was at the time, were still mopping up news from the financial crisis. We were still writing up reports about the toxic loan books that had brought the banks crashing down. The idea of the Government promising to bail out mortgage borrowers seemed the height of recklessness.

But the Government always intended Help to Buy mortgage guarantee to act as a stimulus, not a long-term solution. From the beginning, it had an end date - 31 December 2016. The idea was to encourage big banks to start lending again.

So far, the record of Help to Buy has been pretty good. A first-time buyer in 2013 with a 5 per cent deposit had 56 mortgage products to choose from - not much when you consider some of those products would have been ridiculously expensive or would come with many strings attached. By 2016, according to Moneyfacts, first-time buyers had 271 products to choose from, nearly a five-fold increase

Over the same period, financial regulators have introduced much tougher mortgage affordability rules. First-time buyers can be expected to be interrogated about their income, their little luxuries and how they would cope if interest rates rose (contrary to our expectations in 2013, the Bank of England base rate has actually fallen). 

A criticism that still rings true, however, is that the mortgage guarantee scheme only helps boost demand for properties, while doing nothing about the lack of housing supply. Unlike its sister scheme, the Help to Buy equity loan scheme, there is no incentive for property companies to build more homes. According to FullFact, there were just 112,000 homes being built in England and Wales in 2010. By 2015, that had increased, but only to a mere 149,000.

This lack of supply helps to prop up house prices - one of the factors making it so difficult to get on the housing ladder in the first place. In July, the average house price in England was £233,000. This means a first-time buyer with a 5 per cent deposit of £11,650 would still need to be earning nearly £50,000 to meet most mortgage affordability criteria. In other words, the Help to Buy mortgage guarantee is targeted squarely at the middle class.

The Government plans to maintain the Help to Buy equity loan scheme, which is restricted to new builds, and the Help to Buy ISA, which rewards savers at a time of low interest rates. As for Help to Buy mortgage guarantee, the scheme may be dead, but so long as high street banks are offering 95 per cent mortgages, its effects are still with us.