The Wisdom of Crowds
James Surowiecki Little, Brown, 295pp, £16.99
ISBN 0316861731
Books, like newspapers, require headlines. Life, however, has none. Headlines are thus eye-catching lies or, at the very least, distortions. This may be troubling, but it becomes serious only if the copy beneath partakes of the lie. It is therefore the obligation of the writer to subvert the best efforts of the editor. In this, the New Yorker journalist James Surowiecki has succeeded.
The real headline of this book is in the subtitle: "why the many are smarter than the few". Surowiecki tells the story of Darwin's cousin, the statistician Francis Galton who, in old age, watched a competition to guess the weight of an ox. Eight hundred people took part; some were cattle experts but most were amateurs. Afterwards, Galton collected all the tickets and worked out the average guess. It was 1,197lbs. The ox weighed 1,198lbs. The crowd as a whole had proved more expert than the smartest individual farmer.
Such stories are counter-intuitive because one would expect expertise to be more competent when undiluted by the interventions of non-experts. The revelation that this may not always be so is important. For example, it is now clear that professional investment advisers are no better at forecasting the movements of markets than you or I. Long-term studies have repeatedly shown that professionals tend to do worse than somebody who simply buys the constituents of the FT Index. "With most things," writes Surowiecki, "the average is mediocrity. With decision-making, it's often excellence. You could say it's as if we're programmed to be collectively smart."
However, both markets and fat oxen are special cases. The ox is special because most people at an agricultural show will have quite a high degree of expertise. They will know it weighs, say, more than 500lbs and less than 3,000lbs. Although within this range their estimates may vary wildly, the average, because created by multiple levels of expertise, will tend to converge on the right figure. As such, the wisdom here is not so much of the crowd as of the average. Stock-market prices, meanwhile, are the product of many decisions, made mostly by "experts", all of them possessed of much the same information. It is the very fact that the experts disagree which creates the price; thus, to believe one rather than another is an act of pure faith.
Most things in life are clearly not like that. If I had a leaking pipe, I would call a plumber, not conduct a poll at Homebase. If I wanted to know who should play for England, I would ask Sven-Goran Eriksson, not the shoppers in Oxford Street. Furthermore, the cases in which crowds seem to possess wisdom require a precise cultural context. Markets are sophisticated, highly regulated social products and knowing the rough weight of an ox requires education and experience. The wisdom of crowds is not some Platonic gift from another realm, but an exactly evolved product of human society. The arithmetic of averages helps, but it can kick in only when we have established the ground rules.
Surowiecki, a clear and intelligent writer, knows all this. And so he knows that his book's subtitle is, strictly speaking, inaccurate. In the course of the book the idea becomes so heavily qualified as to be in effect meaningless. However, his real intention, I suspect, is not so much to sell some headline-grabbing idea as to tell some very good stories and to shift the terms of some very high-profile debates.
The stories alone would justify the book. The corruption of Italian football, Ken Livingstone (weirdly misspelt without the final "e") and his congestion charge, the technology of Google, Enron, the space shuttle disaster, the dotcom bubble and many other familiar contemporary hot spots are drawn into the argument. This is high-quality journalism that demonstrates hidden connections in the world and exposes the ideas and illusions by which we are ruled and influenced.
The larger purpose of the book, which emerges towards the end, concerns the workings of government and democracy. At this point a certain intellectual context becomes clear. In the 1990s, in the wake of the collapse of communism, crude neoconservative thought in the US decided that free markets were all that mattered and that the state's primary obligation was to get out of the way. This idea was predicated on a belief in the wisdom of crowds, on the quasi-scientific conviction that, in the right context, people would become rational economic units. The absurd over-simplicity of this idea has now been widely acknowledged.
Surowiecki argues along the same lines. Economic systems do indeed need to be decentralised to encourage local initiative and independence. But there is no point to such decentralisation if there is no strong centre to make it work. In other words, crowds by themselves are not very wise. As I said, good writers subvert headlines.
Representative democracy is a further acknowledgement that crowds may not be wise. We elect delegates whom we require to make detailed and specialised decisions. What we don't do is demand referendums on everything, not just because that would be unwieldy, but also because public opinion is frequently misinformed and always fantastically capricious. An occasional election, however, is like an ox weight-guessing competition; it smooths out error and controls the delusions of experts.
This is, you will gather, a rich and interesting book that sends the mind wandering in many directions. More importantly, it is also a humane corrective to some very damaging contemporary delusions - and that, come to think of it, would make a much more accurate subtitle.
Bryan Appleyard's Understanding the Present is published by Tauris Parke
Post this article to
We want to encourage people to comment on our content and to exchange views with other readers and hope this will be done on a courteous basis. However, if you encounter posts which are offensive please let us know by emailing comments@newstatesman.co.uk and we will take swift action where necessary.
