Before the run on Northern Rock, practically nobody imagined that the banking system could crash. Financial institutions had been showing signs of strain ever since Long-Term Capital Management (LTCM), a highly leveraged hedge fund, blew up in September 1998, nine years before lines of desperate savers began to form outside the offices of the former Tyneside building society. Debt of all kinds had risen to levels beyond anything in living memory. Even so, anyone who suggested that the type of ­turbocharged capitalism released by financial deregulation might be dangerously fragile was dismissed as a doom-monger. Like a successful species that had driven out its evolutionary competitors, hypercapitalism was advancing inexorably throughout the world. How could this Darwinian winner possibly self-destruct?

The failure to perceive the mounting risk of systemic crisis had several sources. Human beings learn too much from the recent past. Despite the meltdown of LTCM, the previous two decades had been a time of apparent calm in the markets. Inevitably, theories emerged - the "long boom", the "end of history" - in which this uneasy interlude was represented as a permanent state of stability. Such theories reinforced the tendency to extrapolate from the near past, but they did more than that. Operating as an ideology, they persuaded many that the unfettered market was not only irresistibly powerful, but also socially benign.

Three years after savers began queuing outside Northern Rock, western finance capitalism is in a worsening crisis, with the bailing out of an insolvent banking system leading to insolvency in government. At the same time, the ideology that legitimated this breed of capitalism is as powerful as ever. As non-executive chair of Northern Rock in the years leading up to its collapse, Matt Ridley can hardly have failed to reflect on the crisis; but there is no sign of him having learned anything from it. He devotes less than a page of The Rational Optimist to the crisis, blaming it on "government monetary and housing policy". The implication is clear: if only governments had not tampered with the market, all would have been well.

As the subtitle of his book indicates, Ridley sees free markets as part of the evolutionary process. This is not evolution of the kind bio­logists understand, however. "Humanity is experiencing an extraordinary burst of evolutionary change, driven by good old-fashioned Darwinian natural selection," he writes. "But it is selection among ideas, not genes." Like Rich­ard Dawkins, another neo-Darwinian missionary, Ridley is a believer in memes - units of meaning that supposedly explain human development. Applying the idea to economics, he writes that "whole economies evolve by natural selection". Just as biological evolution works by bringing together the genes of different individuals, cultural evolution occurs "when ideas meet and mate" in market exchange. “Exchange is to cultural evolution as sex is to biological evolution," he writes. The history of humankind is no more than the working out of this simple equation.

There is nothing new in this kind of thinking. It was the eccentric Victorian sage Herbert Spencer (1820-1903) who coined the expression "survival of the fittest" and promoted the idea that laissez-faire capitalism was the final stage of social evolution. Impressed by Spencer's work, Sidney and Beatrice Webb adopted his idea that economic systems evolve in competition with one another, but nominated Stalinist collectivism rather than the free market as the final winner. Laissez-faire was reinstated as the winning system towards the end of the 20th century, when Spencer's ideology was resurrected in the later writings of Friedrich Hayek. Ridley is doing little more than recycle some of the aged Hayek's dafter ideas.

Whatever political goals it is used to promote, the idea of cultural evolution is not much more than a misleading metaphor. Laissez-faire was not the result of any spontaneous process of social evolution; it was imposed on society through the use of state power. Memes are just a pseudo-scientific way of talking about ideas, not actually existing physical entities. There is nothing in society that resembles the natural selection of random genetic mutations; even if such a mechanism existed, there is nothing to say its workings would be benign. Bad ideas do not evolve into better ones. They tend to recur, as racist memes are doing at present in parts of the world where economic dis­location is reviving hatred of minorities and immigrants. Knowledge advances, but in ethics and politics the same old rubbish keeps on piling up. The idea of social evolution is rubbish of this kind, a virulent meme that continues to reproduce and spread despite having been refuted time and time again.

The best evidence against Ridley's claim that ideas evolve is the existence of this book, which reproduces some of the most pernicious myths of social Darwinism. Spencer and his disciples thought evolution was a progressive movement from lower to higher forms of life. But natural selection has nothing to do with pro­gress - as Darwin put it in his Autobiography, it is like the wind, which blows without any design or purpose. Certainly human development has been affected by the material environment - geography, climate and resource scarcity, for example. But rather than evolving, societies regularly break down, and what comes next is determined by power, chance and (occasionally) human choices rather than any supposed evolutionary laws. Evolution is one thing, progress another, and human history something else again.

Disdainful or ignorant of the past, Ridley is uninterested in the forces that shape events. He writes hundreds of pages about the wealth-increasing virtues of free markets, but allots post-Mao China only a few lines. This brevity is symptomatic, as China falsifies Ridley's central thesis; the largest burst of continuous economic growth in history has occurred without the benefit of free markets. Wealth has been created as never before, not as a result of evolutionary change, but as a product of revolution and dictatorship.

For Ridley, rationality has nothing to do with checking that his beliefs are true. If awkward facts crop up, he ignores them. China is one such fact; another is climate change. He does not exactly deny the existence of global warming, but leaving scientific evidence aside, he invokes the spectre of the world's poor. Developing countries need industrial growth, so global warming is beside the point: "The richer they get, the less weather-dependent their econo­mies will be and the more affordable they will find adaptation to climate change." Here, a demotic appeal to sympathy is combined with dogmatic disregard for real-life conditions. In Africa, the Indian subcontinent and the small Pacific nations, some of the world's poorest societies are already suffering from climate change. Telling them they need more economic growth is not very helpful when they are being destroyed by drought or rising sea levels. In these circumstances, it is Ridley's gung-ho progressivism that is beside the point.

What is needed more than anything else is a clear-eyed view of where we are now and where we might be heading. It is a pleasure to turn from Ridley's febrile visions to Marek Kohn's grimly realistic and yet in many ways inspiring account of how global warming will affect life in Britain. A richly detailed, engrossingly readable history of how Britain came to be the way it is, Turned Out Nice is also a riveting description of what Britain is likely to become. The future Kohn presents is robustly grounded in science, and disturbing. Increased risk of flooding in London and other cities, peak summer temperatures in the capital nearly 7°C hotter than they were in 2000 and inequalities widening further as environmental migrants end up in an expanded servant class - these are only a few of the unsettling changes he anticipates.

The global picture is no less discomfiting. As Kohn writes, "The standard scenarios all confidently expect that wealth will grow along with warmth." In reality, economic development has never been smooth. The growth of wealth has been disrupted regularly by war and revolution, and the rapid recovery that occurred after many 20th-century conflicts will be harder to achieve in a world of accelerating climate change. The conventional wisdom expects that the population will level off around nine billion as a result of higher living standards spread by globalisation. Kohn points to another scenario, in which industrialisation continues while globalisation goes into reverse. In a world of this kind, living standards will rise more unevenly and human numbers will increase to roughly 15 billion.

The picture of the future that Kohn presents is pretty dark, but by no means all grim. Turned Out Nice is full with examples of how we can cope with a shift that can no longer be stopped. We can adjust to intensive city living with "vertical allotments" in stairwells, roof lawns and rejuvenated urban parks. People may be less mobile, but their environment could still be more variegated. Outside cities, the changing climate could be good news for beavers, lynxes and eagles. Britain and the world will be altered, but life will go on.

History is not a process of continuous development, more one of recurrently punctuated equilibrium. In the long sweep of events, there is nothing out of the ordinary in the collapse of Northern Rock, or - in the history of the planet - in global warming. Discontinuity, not gradual change, is the norm. Rather than clinging to flimsy narratives of progress, we need to cultivate the art of intelligent improvisation. That means junking a good deal of rubbish from the past, starting with the idea that free markets are the end point of human evolution.

John Gray is the New Statesman's lead reviewer. His latest book is "Gray's Anatomy: Selected Writings" (Penguin, £10.99)