The Tories' manipulation of education statistics

There is no evidence that reading standards have fallen among school children.

In Saturday's Guardian (Letters, 28 Jan), schools minister Nick Gibb defends the government's view that phonics are the only way to reach children to read. His central justification is that something must be done: "International studies rank England 25th for reading - down from seventh nine years ago."

In the very literal sense, Gibb is correct. In 2000, the OECD’s Programme for International Student Assessment (PISA) placed England in 7th position in its table (p.53). In 2009, it was in the 25th row of a similar table (p. 56).

In any other sense you care to mention, Gibb is entirely wrong , because:

1) Twelve other countries, nominally above England in the 2009 tables, have statistically insignificant higher scores. The National Foundation for Educational Research's summary of the OECD findings is quite explicit about this: "Because of the areas of uncertainty described above, interpretations of very small differences between two sets of results are often meaningless. Were they to be measured again, it could well be that the results would turn out the other way round (p.8)"

2) 31 countries took part in the tests in 2000, and 67 in 2009. Shanghai and Singapore may be nominally above the UK in the 2009 tables but they didn’t take part in the 2006 or earlier surveys. This makes direct comparison between years invalid.

3) The OECD’s warned explicitly (para 2 of this technical note) against comparing earlier PISA results with earlier data, because the very low response rate for earlier years largely invalidated samples.

4) The 2000 and 2003 tests were conducted some months earlier in school year 11 (Nov/Dec) than the 2006/2009 (March-May) ones, as an exception to the international study (to make room for GCSE preparations). As John Jerrim of the Institute of Education has noted, taking the tests around half a school year early makes a very obvious difference: "[I]t is important to understand that between November/December and March‐May of year 11 is likely to be a period when children add substantially to their knowledge of the PISA subjects as it is when pupils are working towards important national exams. Consequently, one should expect the year 11 pupils in the PISA 2000/2003 cohort to out‐perform their peer taking the test in 2006/2009 due to the extra five months they have had at school….."

In short, there is simply no reliable evidence that 15-year-olds in England are any less able to read and understand texts, when compared to their international peers, than they were nine year ago. Yet here we have a government minister using that argument as a key reason for a fundamental and controversial change in which five and six-year-olds are taught.

Now, if this was a result of incompetence on the part of the minister and his department, that would be worrying enough. But what should really concern us is that the Department of Education almost certainly knows perfectly well that its "interpretation" of the OECD data is entirely incorrect, but is determined to carry on peddling its untruths anyway.

The key evidence of this, I suggest, is the way in which Michael Gove himself defended his proposals for a return to 'O' Levels/CSE in parliament on 21 June:

The sad truth is that, if we look at the objective measure of how we have done over the past 15 years, we find that on international league tables our schools fell in reading from 523 to 494 points, in maths from 529 to 492 and in science from 528 to 514.

Here, Gove used the OECD raw scores for 2000 and 2009 rather than the table rankings (the lower scores can largely still be explained by two of the factors above). He almost certainly did this because he and his team realised they had been rumbled by blogs like Though Cowards Flinch with a mind to detail, and by a Guardian editorial of the same day, which said:

Mr Gove.... latches on to data purporting to show English schools plummeting down world rankings. The Institute of Education has meticulously documented all sorts of distortions in these apparently alarming figures, but such calming analysis fails to register. Mr Gove should go away, revise the evidence properly – and prepare for a resit.

Clearly, Gove didn't want to be caught red-handed by Labour members assiduous enough to have read the Guardian that morning. Yet just a month later we have the schools minister writing to the same paper with the very nonsense his boss had been wary of using.

The real tragedy, of course, is not that Guardian readers are being lied to, but that actual educational policy is being developed on the basis of false data. The direct consequence of the pretence that comparative reading standards are plummeting is a emphasis on setting higher targets, as set out by Sir Michael Wilshaw, the Chief Inspector of Schools who, sadly, has been all too complicit in the myth-making. Wilshaw has stated that: "So one of the first questions we need to ask is whether the national end-of-primary-school target of level 4 is sufficiently high to provide an adequate foundation for success at secondary school."

Yet data in the government's own 2010 education white paper suggests that the actual problem policymakers should be facing up to is not low targets, but unequal distributi on of achievement between the upper and lower percentiles compared with other countries (see Exhibit 1.1 in this PIRLS report). By focusing their energies on the creation of fundamentally dishonest headlines, the government and its advisers are actively missing out on data which might actually improve the lives of young people.

Of course, this is not the first time that the government has resorted to the use of dodgy statistics. Chris Grayling has already had his wrists slapped by the UK Statistics Authority for his flagrant abuse of statistics. Now, it even looks as though the government may attemp to explain away its disastrous management of the economy by casting doubt on the reliability of the GDP data collected by the Office for National Statistics, without providing a shred of evidence as to how these dataset might have been considered reliable for so long but are now, so suddenly, suspect.

Overall, a picture is starting to emerge of a government prepared, in its mix of desperation and ideological fervour, to go one step beyond spin. That should keep us on our toes.

"Actual educational policy is being developed on the basis of false data." Photograph: Getty Images.

Paul Cotterill is a blogger for Liberal Conspiracy and Though Cowards Flinch.

Photo: ASA
Show Hide image

Harmful gender stereotypes in ads have real impact – so we're challenging them

The ASA must make sure future generations don't recoil at our commercials.

July’s been quite the month for gender in the news. From Jodie Whittaker’s casting in Doctor Who, to trains “so simple even women can drive them”, to how much the Beeb pays its female talent, gender issues have dominated. 

You might think it was an appropriate time for the Advertising Standards Authority (ASA) to launch our own contribution to the debate, Depictions, Perceptions and Harm: a report on gender stereotypes in advertising, the result of more than a year’s careful scrutiny of the evidence base.

Our report makes the case that, while most ads (and the businesses behind them) are getting it right when it comes to avoiding damaging gender stereotypes, the evidence suggests that some could do with reigning it in a little. Specifically, it argues that some ads can contribute to real world harms in the way they portray gender roles and characteristics.

We’re not talking here about ads that show a woman doing the cleaning or a man the DIY. It would be most odd if advertisers couldn’t depict a woman doing the family shop or a man mowing the lawn. Ads cannot be divorced from reality.

What we’re talking about is ads that go significantly further by, for example, suggesting through their content and context that it’s a mum’s sole duty to tidy up after her family, who’ve just trashed the house. Or that an activity or career is inappropriate for a girl because it’s the preserve of men. Or that boys are not “proper” boys if they’re not strong and stoical. Or that men are hopeless at simple parental or household tasks because they’re, well...men.

Advertising is only a small contributor to gender stereotyping, but a contributor it is. And there’s ever greater recognition of the harms that can result from gender stereotyping. Put simply, gender stereotypes can lead us to have a narrower sense of ourselves – how we can behave, who we can be, the opportunities we can take, the decisions we can make. And they can lead other people to have a narrower sense of us too. 

That can affect individuals, whatever their gender. It can affect the economy: we have a shortage of engineers in this country, in part, says the UK’s National Academy of Engineering, because many women don’t see it as a career for them. And it can affect our society as a whole.

Many businesses get this already. A few weeks ago, UN Women and Unilever announced the global launch of Unstereotype Alliance, with some of the world’s biggest companies, including Proctor & Gamble, Mars, Diageo, Facebook and Google signing up. Advertising agencies like JWT and UM have very recently published their own research, further shining the spotlight on gender stereotyping in advertising. 

At the ASA, we see our UK work as a complement to an increasingly global response to the issue. And we’re doing it with broad support from the UK advertising industry: the Committees of Advertising Practice (CAP) – the industry bodies which author the UK Advertising Codes that we administer – have been very closely involved in our work and will now flesh out the standards we need to help advertisers stay on the right side of the line.

Needless to say, our report has attracted a fair amount of comment. And commentators have made some interesting and important arguments. Take my “ads cannot be divorced from reality” point above. Clearly we – the UK advertising regulator - must take into account the way things are, but what should we do if, for example, an ad is reflecting a part of society as it is now, but that part is not fair and equal? 

The ad might simply be mirroring the way things are, but at a time when many people in our society, including through public policy and equality laws, are trying to mould it into something different. If we reign in the more extreme examples, are we being social engineers? Or are we simply taking a small step in redressing the imbalance in a society where the drip, drip, drip of gender stereotyping over many years has, itself, been social engineering. And social engineering which, ironically, has left us with too few engineers.

Read more: Why new rules on gender stereotyping in ads benefit men, too

The report gave news outlets a chance to run plenty of well-known ads from yesteryear. Fairy Liquid, Shake 'n' Vac and some real “even a woman can open it”-type horrors from decades ago. For some, that was an opportunity to make the point that ads really were sexist back then, but everything’s fine on the gender stereotyping front today. That argument shows a real lack of imagination. 

History has not stopped. If we’re looking back at ads of 50 years ago and marvelling at how we thought they were OK back then, despite knowing they were products of their time, won’t our children and grandchildren be doing exactly the same thing in 50 years’ time? What “norms” now will seem antiquated and unpleasant in the future? We think the evidence points to some portrayals of gender roles and characteristics being precisely such norms, excused by some today on the basis that that’s just the way it is.

Our report signals that change is coming. CAP will now work on the standards so we can pin down the rules and official guidance. We don’t want to catch advertisers out, so we and CAP will work hard to provide as much advice and training as we can, so they can get their ads right in the first place. And from next year, we at the ASA will make sure those standards are followed, taking care that our regulation is balanced and wholly respectful of the public’s desire to continue to see creative ads that are relevant, entertaining and informative. 

You won’t see a sea-change in the ads that appear, but we hope to smooth some of the rougher edges. This is a small but important step in making sure modern society is better represented in ads.

Guy Parker is CEO of the ASA