Great figure - but can you trust it?

Observations on polls

On 15 September, the Sunday Times reported growing support among the British public for an attack on Iraq. Though 57 per cent would oppose Britain contributing troops if the US decided to go it alone, 80 per cent would back UN action if Saddam refused to re-admit the inspectors. But should we believe these results? They were gathered from a sample of 2,115 people - more than twice the numbers you would normally expect for a newspaper poll - questioned by YouGov, the increasingly popular internet polling service.

Can a medium that has such a partial distribution - the internet is accessed by fewer than 50 per cent of the adult population, with users heavily skewed towards younger middle-class males - be used to track the opinions of the British people?

The answer lies in the magical word "weighting". YouGov asks its respondents about their age, gender, past voting behaviour, newspaper-reading habits, occupation and so on. They then compare this with what is known about the national population as a whole and re-weight the results accordingly. Suppose the percentage of the British population aged 65 and over is 20 per cent, but the over-65s amount to only 5 per cent of the YouGov sample. The pollsters re-weight the answers of that 5 per cent to make them worth 20 per cent in the final results.

Most pollsters use re-weighting to some degree, but the general wisdom is that the less done, the better. For internet polling, you need lots of re-weighting. Worse, your re-weighting is only as good as your data about the general population, and there is no data yet from the most up-to-date source of information about the British population - the 2001 census. Worse still, it is impossible to check that an internet respondent who says that she is an elderly woman living in Birmingham isn't, in fact, a teenage boy in Bermondsey.

The YouGov sample is made up of people who have volunteered to be part of its pool and thus are not typical of the population as a whole. They are also paid, which introduces another source of possible bias. So does the concept of the panel - a constant group to whom pollsters return at intervals. It is well known that as panel members become accustomed to being polled, they become less typical of the wider population .

Finally, the model assumes that the traditional data about age, gender, class and so on are still reliable guides to people's voting behaviour, when there are many indications that this is no longer the case.

Yet YouGov can point to its success in predicting the outcomes of the 2001 general election, the Pop Idol contest and other national events.

Polling by internet is fast, cheap and reaches parts of the population that other methods can't. But as for its accuracy, I have to say, as so many respondents do to traditional pollsters: "Mark me down as a Don't Know."

The New Media Awards 2002 winners will be presented with their trophies on 1 October at the Hilton, Blackpool


Next Article