The fate of the oil under the Caspian Sea illustrates how the worlds before and after the Russian revolution appear to join up, as if Soviet rule were a hiatus and history has taken up where it left off as the Bolsheviks triumphed in Moscow. At the beginning of the 20th century, the oilfields of Azerbaijan made Russia the world's biggest oil producer. The boom brought investors from Europe to the Azerbaijani capital, Baku, where they lived in the plush mansions near the shore of the Caspian, built from the fortunes that oil brought.
The streets of the old town are still overlooked by ornate balconies, vines curled around their railings. Decades of Soviet privation have not erased a certain grandeur. Azeris like to boast that the first opera house in a Muslim country opened in Baku. After the revolution, Azerbaijan was absorbed into the Soviet Union and the oilfields of the Caspian were largely neglected. After Azerbaijan won its independence in 1991, the international oil companies again descended on Baku.
Traces of the old Baku and the bold intrusions generated by the repeat oil boom can be seen side by side in the historic town centre. From the top floor of a Soviet-era hotel overlooking the seafront, you can see ancient, rusty oil rigs out to sea. Behind the promenade is a new hotel with a penthouse bar where American engineers and geologists sip cocktails. Down charming streets, alongside clothes shops and delicatessens, are the Irish, Scottish and English theme pubs where the oil rig workers drink. It is easy to believe you are in Dundee or Dunstable.
This summer, Baku was abuzz with its annual oil conference, which takes place in an indoor sports stadium where stands exhibit the essentials of the oil business: pumps and compressors, drilling tools and submersible suits. Germans, Russians, Britons, Frenchmen and Americans mill around in suits. Billboards with obscure acronyms and glossy brochures extol the competence of companies you've never heard of, with a sprinkling of familiar names such as Exxon Mobil and BP.
In 1994, the government of Azerbaijan and a consortium of international oil companies signed a contract they called "the deal of the century". Expectations of the size of the Caspian reserves were high and President Bill Clinton decided the oil would be a perfect alternative to Middle East supplies. Clinton's main concern was a new pipeline to bring the oil out. He didn't want any more oil to run through the Russian pipelines already there. And he especially wanted to keep Iran isolated. A pipeline through Iran - which shares a long border with Azerbaijan - would make geographic and commercial sense, and the Iranians had designs on exercising their influence as the major regional power. But the Americans wanted Turkey to play that role and so their preferred route was from Baku, west to Tbilisi, the capital of Georgia, and then into Turkey, ending up at the Mediterranean port of Ceyhan. Clinton had to work hard on the governments and on the oil companies, who doubted the pipeline's commercial viability.
Since the early 1990s, Azerbaijan has been ruled by President Heydar Aliyev, a Brezhnev-style member of the last Soviet politburo who had transformed himself into the champion of Azerbaijani independence, a firm ruler who promised to protect the country from the Russians as well as from its own potential for disorder. Aliyev is now gravely ill, close to death. But his face stares from the entrance to every ministry in Baku and his slogans and exhortations shout from billboards on major roads. He won his presidency in elections, but no foreign observers have said they were free and fair. In Baku, I met Vafa Guluzade, Aliyev's foreign policy adviser during the 1990s. Now retired to a small dacha outside the capital, he is eager to talk of the days when he met one Washington luminary after another. Guluzade would take Clinton's calls and translate them for Aliyev. Clinton was pushing the idea of "multiple pipelines" - the route through Georgia to Turkey. Aliyev decided it was in Azerbaijan's interest to go along with Clinton. Eventually, the oil companies, led by BP, were brought around to the idea of the pipeline, helped crucially by the announcement by Turkey (America's ally) that it would give a large subsidy to the $3bn project.
Finally, pipe-laying has begun - a triumph of relentless American diplomatic efforts in pursuit of ostensibly commercial ends. But for the US, access to Azerbaijan's oil has not been just about commerce. The American presence in Baku, made natural by the quest for oil, is transforming local geopolitics. For Azerbaijan, moving close to the US is a way of getting away from the Russians. Suspicions that Moscow wants Azerbaijan back, or at least to undermine its independence, are still aired in Baku. Stephen Mann, President Bush's special envoy to the Caspian region, whose job is to ensure the pipeline gets built by sorting out difficulties between the various governments and between the countries and the oil companies, talks about consolidating the independence of the countries around the Caspian. It really means moving the countries from the Russian to the American sphere of influence.
The events of 11 September 2001 brought home to the Americans how useful it would be to have allies in the Caspian. Guluzade remembers trying to persuade a Pentagon official a few years ago of Azerbaijan's strategic importance: its long border with Iran, its proximity to Chechnya. It was not time yet, the Pentagon man told him. Now the time has come. When war in Afghanistan began, US military planes were given the right to fly over Azerbaijan. The potential for instability in the states near the Caspian has driven the US to increase its military commitment.
Before BP allowed me to look at its pipeline construction outside Baku, I was given an extensive safety briefing, mainly to make sure I didn't walk under the machinery or get bitten by snakes and scorpions. But one part of the briefing really brought home the troubled territory this pipeline will pass through: a map marked with several no-go areas for BP employees. One such area is the small valley in Georgia called the Pankisi Gorge, where Chechen rebels have taken over a series of villages. The BP pipeline from the Caspian to Turkey will pass within 60 miles of the Chechen positions. It's an obvious target for guerrillas and last year US troops arrived in Georgia to train local forces to fight the Chechens. Just after I returned from Baku, the Pentagon announced it would be sending troops to Azerbaijan. They will come from Germany, a sign that after more than a decade of drift since the end of the cold war, a new world order is finally taking shape: one where the US is moving across the frontiers created by the Second World War and making its presence felt in parts of the world Washington previously ignored. General Charles Wald, the deputy commander of the US European Command, was explicit about the new mission: the aim, he said, was to protect the long-term viability of oil and gas reserves in the Caspian.
The Americans are performing in the Caspian the kind of job that the British would once have done - though in a mobile, unbuttoned, non-imperial style suited to its distaste of the old European powers that once dominated the world. The most potent symbol of the transfer of power from London to Washington at the end of the Second World War was the tussle over control of the oil in the Middle East. In 1944, President Roosevelt summoned the British ambassador to the White House. The president explained his idea of a postwar settlement on a rough map of the Persian Gulf he'd sketched out himself in the Oval Office. The British could have Iran, the oil of Iraq and Kuwait would be shared, but Saudi Arabia, Roosevelt declared, "is ours". In March 1944, Churchill wrote to Roosevelt in an attempt to calm the dispute over oil. "Thank you very much for your assurance about no sheep's eyes at our oilfields in Iran and Iraq. Let me reciprocate by giving you fullest assurance that we have no thought of trying to horn in upon your interests or property in Saudi Arabia."
Much has changed. The dominance of the international companies ended with the nationalisations of the 1970s, when oil producers took control of their resources. But securing supplies is still a key strategic US objective and those with oil to sell know it.
In one of his many anecdotes about his time at the centre of power in Baku, Vafa Guluzade told me how President Aliyev once explained the realities of the post-cold war world in terms of the calculation of someone schooled in Soviet realpolitik. "Vafa," Aliyev said, "do you see that now the politburo is not in Moscow, it is in Washington."
Maurice Walsh presents a four-part series on oil and US foreign policy on BBC World Service beginning on 15 September