Henry Kissinger, in 1982, wrote: “Blessed are the people whose leaders can look destiny in the eye without flinching but also without attempting to play God.” The former US secretary of state is an unlikely – and unfashionable – source of reassurance, but his injunction is one that the west would do well to follow in the Obama era.

The US National Intelligence Council predicted a bleak future in its most recent Global Trends Review. America's dominance will disappear by 2025, it said, and the EU will become a "hobbled giant", unable despite its economic strength to exert significant global influence. With the last superpower reduced to a "first among equals" as new giants rise in the east, the "unipolar world" will be "over". The report warns of nuclear proliferation, mass migration, environmental catastrophe. "The next 20 years," it says (just to make sure we've all got the point), "are fraught with risks." Confronted with these dangers and uncertainties, however, some western leaders are still overly tempted to "play God".

At the Munich Security Conference on 7 February, Kissinger was awarded the first Ewald von Kleist prize for his "contributions to global peace and international co-operation". At the same time reports emerged that President Barack Obama had sent the good doctor to conduct secret talks on nuclear weapons reduction with Moscow in December.

“We cannot rule out arms races, territorial expansion and military rivalries”

But the world leaders gathered in Munich also heard the first major address on the new administration's foreign policy. Although Vice-President Joe Biden spoke softly - "We'll engage. We'll listen. We'll consult" - he still carried a big stick, delivering warnings to Russia and Iran, and urging US allies to be more willing "to use force when all else fails". His remarks were consistent with Secretary of State Clinton's statement at last month's Senate confirmation hearings, when she denied reports of her country's imminent relegation to equal rank status with other world powers: "Some have argued that we have reached the end of the 'American moment' in world history. I disagree."

Hillary Clinton advocated the use of "smart power", combining "hard" military and economic with "soft" cultural and diplomatic tools. That may sound eminently reasonable, but let's note how the Bill Clinton-era diplomat Suzanne Nossel concluded the essay in which she popularised the term in 2004: "Now is the time . . . to reassert an aggressive brand of liberal internationalism . . . and fortify it through the determined, smart use of power."

Such talk of aggression is dangerously misplaced. The chaotic, uncertain world of today requires something starkly different. It is time, instead, for a new realpolitik.

In one sense, realpolitik never went away. Its cardinal principle of non-interference - that no state has the right to intervene in the internal affairs of another - is one to which over half of humankind is theoretically signed up, through the 118 countries that belong to the Non-Aligned Movement (NAM). The developing-world titans who founded it in 1961 - Nasser, Nkrumah, Nehru, Tito and Sukarno - are long gone, and we in Britain may hear little of the NAM. But it goes far from unnoticed in the United States, not least because Cuba (under Raú Castro) holds the presidency of the organisation and Hugo Chávez emerged as the star of its last summit in 2006. It regularly votes as a bloc at the UN General Assembly, as do other caucuses of developing countries such as the Group of 77. In an interview late last year, Noam Chomsky dismissed suggestions that the NAM was a relic of the Cold War. "I think that it is a sign of the future," he said.

The more recently formed Shanghai Co-operation Organisation is another body of which we hear little. But perhaps we should pay more attention. Made up of Russia, China and four former Soviet central Asian republics, the SCO clearly states non-interference as a core principle in its charter - as does Asean, the ten-country Association of South-East Asian Nations, whose combined population is close to 600 million.

Admittedly, realpolitik has sometimes been used to symbolise the very opposite. In association with Kissinger, for instance, it has come to stand for all the excesses of US foreign policy during the period he served as national security adviser and secretary of state under Presidents Nixon and Ford.

This is to cast the doctrine purely (and thus falsely) in terms of the cold pursuit of national interest (often masquerading under the cover of "spreading freedom") that led some to charge Kissinger with war crimes. It obscures the great successes of his realpolitik: détente with the Soviet Union, the opening of relations with China, and the shuttle diplomacy that ended the Yom Kippur War and ultimately laid the foundations for Jimmy Carter to host the Camp David Accords between Egypt and Israel.

It is this pragmatic aspect of Kissinger's foreign policy that should inform a new realpolitik. Yes, the human rights records of many of these states was lamentable and scruples were understandable. Yet the outcome was increased peace and stability. Was that not a greater prize than a salved conscience?

"What the realist fears is the consequences of idealism." The words belong to Brent Scowcroft, national security adviser under the first President Bush and a disciple of Kissinger. Their conservative provenance should not stop us from recognising that if only they had been engraved in brass and placed on the desk of every foreign minister in the west we might have been spared much dangerous posturing over the past decade.

It was foolish idealism that led to Nato's eastward expansion into the new democracies of the old Soviet bloc. (One assumes so, since no Nato partner rests more easily in his bed knowing that the might of Latvia and Lithuania is now at his disposal.) The realist would have pointed out that this humiliation of Russia, in the process encircling its Baltic exclave of Kaliningrad, was perhaps not the best way to build friendlier relations with the possessor of the world's largest natural gas reserves. Nor that announcing plans to instal interceptor missile bases in Poland and the Czech Republic would be taken in particularly good part.

Russia's reactions, both in Georgia and to the missile bases, should have been expected. Dmitry Medvedev will not be the last occupier of the Kremlin to defend his country's "privileged interests" in neighbouring states: the demise of the USSR did not excise centuries of Russian domination from the history books, nor from that nation's sense of self.

Idealism of a different hue bedevils the west's relations with China. Today, Hollywood film stars in thrall to a media-savvy old monk have encouraged many to regard the patient diplomacy that led to Richard Nixon's breakthrough as pusillanimous gradualism; public pressure and face-shaming demonstrations are seen as the way to persuade Beijing to act over Tibet. (Not having the benefit of such good-looking advocates, other regions with equally worthy claims to greater autonomy are apparently of little concern.) Barack Obama's voice was raised in the idealistic campaign to boycott the Beijing Olympics last year. Reality has since bitten, and he must hope the Chinese are willing to overlook his part in that shouty chorus, now he needs them to bail out the US economy.

Go to Riyadh, Singapore or St Petersburg, and you will find populations deeply convinced of differing value systems. Idealistic liberal internationalists, however, see superficial similarities – a Norman Foster building in Shanghai, a McDonald’s in Cairo – and assume that sharing consumer culture leads to a common political culture. We are entitled to hope that that will happen, though we would be wise to follow Scowcroft’s advice about how to help the process: “You encourage democracy over time, with assistance, and aid, the traditional way. Not how the neocons do it.” We have no reason, however, to shade our hope into certainty.

We should also acknowledge that in the past 30 years Wahhabist Islam has been far more successful at exporting itself, at the expense of pre-existing, liberal political cultures in Muslim countries, and often through the precise means Scowcroft suggests: funding hospitals, schools and the like.

If anything, our era is marked by the reassertion of older, less globally unifying impulses. "We cannot rule out a 19th-century-like scenario of arms races, territorial expansion and military rivalries," concludes the NIC report, which also suggests that several African countries may become completely ungovernable.

Such forecasts bode ill for the inevitable progress of liberal universalism. Yet so does the unacknowledged reality of the present. You do not have to share Chomsky's optimism about the Non-Aligned Movement as an organisation, for instance, to appreciate the long-term significance of its support for Iranian nuclear enrichment. "The fact of the matter is that the majority of the world supports Iran," he pointed out. "But they are not part of the world, from the US point of view." It is a view that can be sustained as long as the west has overwhelming superiority in wealth and weapons. What happens when it doesn't?

Sooner or later China, Russia and that "rest of the world" we ignore, except to luxuriate on its beaches or to shed a tear for its natural disasters, will demand that we meet them on their terms, and not just ours. This will be no surprise to Kissinger-era diplomats, who knew that history's arc was uncertain and quite possibly endless, and that there are many painful questions to which there are no satisfying answers, just a series of "least worst" options.

Realpolitik may not offer the comfort of doing the "right thing". However, until we can agree on what the "right thing" is, that is a moral discomfort we must learn to bear. If the alternative requires shackling, or bribing, or threatening our fellow man to concur, there is nothing "smart" about it.