In his last years, Willie Whitelaw began to worry that there would be serious trouble over fox-hunting. He couldn't quite see why, but he thought it could become an issue that would bedevil the government of the country. The frail old grandee knew a thing or two about British politics. He had spotted a shift that would wrongfoot all the mainstream parties. The old agenda - on public spending, taxation, the role of the state - is no longer at the centre of politics; but it has yet to be replaced by one that accurately reflects public values.

Fox-hunting is only one of a number of newly pivotal issues that have confounded the political classes. Why did the government get public opinion so badly wrong on GM food? Why are campaigning groups so much more effective than political parties in setting the public agenda? What is it about politics in the nineties that the politicians have failed to understand?

As with so many questions about the nineties, the answers lie in the eighties. By the close of that far-off decade, many otherwise sensible people were convinced that the end of politics had arrived. In the post-cold war era, they believed, it was not ideas that decided the fate of governments, but competence - above all, in running the economy. "It's the economy, stupid!" became the watchword of politicians throughout the nineties.

The real lesson of the nineties, however, is "It's not the economy, stupid!" The issues that rallied public opinion, derailed flagship policies and helped to destroy governments during the nineties were not chiefly economic. They had more to do with risk, trust and the quality of life. The penny has yet to drop for Britain's political classes that their fortunes will largely depend on how they handle this new agenda.

No party can hope for office if it is seen as incompetent in managing the economy. Even so, the fashionable notion that we live in a post-political era, in which government is about little more than delivering prosperity and putting a favourable spin on events, is not much more than a half-truth.

John Major won the 1992 election in the middle of a recession. It was his government's paralysis during the BSE crisis, its reputation for tolerating sleaze and its divisions over Europe, as much as its handling of Black Wednesday, that doomed it to defeat in 1997.

British voters are surely interested in growing steadily richer. But as long as they remain confident of prosperity, economic management is no longer at the heart of what they want from government. Their chief concern is that government should respond effectively to their worries about new risks and threats to the quality of their lives.

In this, British voters are at one with those of most other affluent late-modern countries. Why, then, do the mainstream parties continue to consign these issues to the peripheries of serious politics? The reason, I believe, is that they are ruled by a dated understanding of modern society. Although they are committed to modernising the country's institutions, the political classes have a view of the values of late-modern Britain that is in crucial respects anachronistic.

The Conservatives are the ones most palpably adrift. It is not so long ago that Tories were talking sagely of Britain's "quiet majority" - that mass of Pooterish folk, ignored or derided by the liberal intelligentsia, who supposedly shared John Major's nostalgia for "traditional values". After the last election and the public response to the death of Diana, the Tories have had to face the awkward truth that this conservative majority no longer exists.

Neither the Tory party nor what is left of the Conservative intelligentsia knows what to make of this. If there was ever such a thing as a "Tory philosophy", it had to do with the belief that wise governments align themselves with the prejudices of the majority. It was taken for granted that these prejudices were conservative.

For Conservatives, the discovery that the prejudices of the majority are nowadays mostly liberal is deeply disorienting. Under William Hague's leadership, the Tory party shows little interest in reclaiming the ground it has lost. In a parodic replay of the Labour left's "rainbow coalition" in the early eighties, the Tories are reinventing themselves as a reactionary counter- culture, a mud-coloured coalition of fox-hunters, gun-owners, Little Englanders, eccentric traditionalists and wild-eyed libertarians. The clear danger is that the Tories will end up as the party of alienated minorities that do not feel at home in the modern country that Britain has become.

By contrast, Labour has gone out of its way to identify itself with modernity - though its talk of modernisation sometimes has a slightly tinny ring. The government is entirely right in its belief that new technologies are changing our lives irreversibly, and to insist that information technology is at the heart of economic renewal. The government's initiatives on entrepreneurship and lifelong learning are intelligent responses to the way the Internet is making the old way of organising working life in a lifelong career unviable. Even so, being modern does not mean embracing any and every new technology. It means testing new technologies against modern values.

There is nothing anti-modern about suspicion of GM food. It is of a piece with the enhanced awareness of risk and concern for the environment that goes with affluence. The public is patently worried about some new technologies, and it is unwilling to accept the judgement of scientists regarding the risks simply on trust. Nothing is easier than to dismiss this public response as hysterical, ignorant or Luddite - just the kind of thing that a modernising government should seek to counter and root out. Such attitudes are precisely what one should expect in rich, technology-based societies.

As most people's basic needs are met, it is natural that they should become more concerned with the integrity of the environment. As the way they live becomes ever more bound up with new technologies, it is reasonable that they should demand that their risks be rigorously monitored. And as they become more wedded to making their own choices, they are increasingly dissatisfied with governments that disparage their lay judgements of risk by appealing to the authority of a (sometimes imaginary) scientific consensus.

Although the British majority will not put up with moralising politicians telling them how they should lead their personal lives, it insists that it is the government's responsibility to protect the common environment. The failure of politicians in all the main parties to understand the values of voters has a common root in a redundant idea of what it is to be modern. The political classes are bewitched by a positivist view of modernity that dates from the start of the 19th century. Saint-Simon and Comte believed that technological and scientific development would make human life more controllable - and thereby safer. They identified social progress with the growth of scientific knowledge and the use of technology to master the Earth. This view had an immense impact on European thought. Through its influence on Marx and the early Fabians, it helped to shape the outlook of the left.

Yet the positivist vision of modernity is narrow and in some ways plainly mistaken. The positivists and their followers did not anticipate that as our lives became more malleable by technology, they would also become in some respects riskier. They were confident that as modern societies came to depend on scientific knowledge, their values would increasingly be shaped by science and technology. It did not occur to them that belief in the intrinsic value of the natural world would be most intensely felt in societies that had reached the highest levels of technological development. They did not foresee that as societies became more modern, their values would become more politically contested.

The belief that we live in a post-political age is a reflection of a discredited idea of modernity. The broad consensus that has been reached on the importance of a well-managed market economy has not produced a pragmatic style of politics in which all that matters is competence. Instead, other issues have been politicised, and politics has moved outside the control of political parties. In part this is a benign development, as it has enabled campaigning organisations such as Friends of the Earth to shape policy in a way that better reflects public values. But it has a darker side. The politics of values can turn poisonous.

The failures of the old parties create an opportunity for a new breed of populist. They provide an opening for politicians whose chief appeal comes from being perceived as not belonging to the political classes. This is the public image which Pat Buchanan is crafting for himself as a presidential candidate in the United States, and it is the same image as a political outsider that has allowed the far-right Jorg Haider to emerge as a potential maker and breaker of governments in Austria.

In Britain, Ken Livingstone's campaign to be mayor of London thrives because the public does not see him as a professional politician like his rivals. This impression has no basis in Livingstone's history, which is that of a professional politician who made his way through the old Labour machine. The public perception of him as a political outsider is an artifice, a product of his skilful use of the media. But Livingstone does connect with public concerns in a way that the government - along with the rest of the political mainstream - does not. Putting the appalling condition of transport in London at the centre of his campaign gives him an advantage. At a stroke it enables him to focus the public's attention on the high-point in his political career and a major blight in the quality of their lives. If he becomes mayor, Livingstone will surely not solve London's transport problems. But he will show how wide the gap has become between public values and the political mainstream.

Politics did not come to an end in the nineties; it simply moved on. Willie Whitelaw was a long way ahead of the game in observing how an issue that had long been viewed as being on the margins of politics could suddenly mark out a new terrain of serious conflict. Perhaps a setback in the world economy will revive the old bread-and-butter politics of "It's the economy, stupid!". But the new agenda of risk and the quality of life will not go away. If it continues to be sidelined by the political classes, it will re-emerge where the real action in politics often takes place today - in the struggles of campaigning organisations and the machinations of media-savvy populists.

John Gray is professor of European thought at the London School of Economics