The forthcoming Lord of the Rings film trilogy is a risky venture. Tolkien fans have been working themselves up about it for months, and if it betrays their sacred text, they won't be happy. The childhood magic of Harry Potter is one thing, but it might be a little harder to make us take seriously a world in which dwarves, elves and orcs fight and die. One of the biggest problems facing the director of the trilogy, the New Zealander Peter Jackson, is that he is also competing with the most successful series of movies in the past three decades, the Star Wars films. The next in that series, Attack of the Clones, will be released shortly after The Fellowship of the Ring, but Star Wars has already done more than enough to hamstring its rival. It has made it overfamiliar.

The story is the same. The disparate, hugely outgunned forces of good fight a hopeless battle against the gigantic evil empire. The unlikely young hero, Frodo Baggins or Luke Skywalker, is tutored by the old bearded magician, Gandalf or Obi Wan Kenobi, both of whom die and return. The hero invades the inner sanctum of the dark forces, Mordor or the Death Star, and destroys it. In both, a mystical power - the Ring or the Force - is capable of good or evil.

The similarities are not coincidental. Both stories are shaped by the same mythology. In writing Star Wars, George Lucas was influenced by the theories of Joseph Campbell about mythic archetypes. Tolkien, by contrast, did his research himself, deriving his stories from Arthurian, Norse, Celtic and classical myth, mixing in some Christianity for good measure.

But The Lord of the Rings was shaped not just by myth, but by the decline of the British empire. Written between 1936 and 1949, it emerged in the aftermath of the two world wars, which left the British empire victorious but in terminal decline. Tolkien himself fiercely dismissed any attempts to interpret his stories as allegories, but it is impossible not to see, in the forces of Mordor, the growing strength of Germany and its imperial ambitions. Even the course of the war in Middle-earth, with opposition to Mordor disunified, ineffective and unable to do more than fight a desperate and seemingly doomed defence, sticks very closely to the British experience of the first three years of war against Germany.

The identity of the hobbits is also pretty clear. Their home, the Shire, is a simple rural place. They are quiet people who like their food and drink, but are fearsome when roused to anger. The British retained this rather unlikely self-image during the period of empire. Although the first great industrial power, with dominion spread around the world, they wanted to believe that their essential nature was one of straightforward country people. They lived in a land of warm beer and cricket, relying on simple virtue for the victories they won.

It has been argued that Tolkien's fascination with Norse myth is worryingly close to the Nazis' obsession with all things Aryan. The elves are a little too blond and perfect, a little too compromised when they mix their blood with humans. We need to pursue the allegory to answer this. Tolkien lets us know that the orcs, the most evil race of all, are themselves elves, experimented on and twisted to become the soldiers of Mordor. This is far from flattering as an image of the German master race. And where are the Jews? If my identification is right, Tolkien gives a familiar caricature of them. They are the dwarves, peculiar-looking, with an insatiable love of gold. The image is not exactly politically correct, but the dwarves are firmly on the side of good. In the developing relationship between Legolas Greenleaf, the perfect Aryan elf, and the dwarf Gimli, Tolkien offers a vigorous alternative to the policies of the Third Reich.

Beyond the echoes of the Second World War, The Lord of the Rings is permeated by another strikingly British element - an insidious pessimism about the future. There is a continual sense of Middle-earth as a place in decline. The elves, the incarnation of a better, older time, are having less and less to do with us. Soon they will leave Middle-earth altogether. The ents, the spirits of the trees and nature, are growing older and will soon vanish, too.

Peter Jackson has talked about his desire that his cinematic version of the books should incorporate their sadness and sense of loss. He suggests that a Hollywood director might not have been able to cope with such nuances. Certainly, when George Lucas put very similar mythic structures on the screen in Star Wars, there was none of the sadness that runs through The Lord of the Rings. Although the story of Star Wars is set in a galaxy "long, long ago", there is no nostalgia for a vanished past. There is also no sense that victory is anything but good. Star Wars itself, Return of the Jedi and The Phantom Menace all end with medals ceremonies and unmitigated triumphalism.

This difference in attitudes reflects the different outlooks of the British and American empires. Britain, France, Holland, Spain, Portugal, Italy and Greece are all now small nations living in the shadow of a lost dominion. But the US remains the only western nation never to have experienced decline from a former golden age. Even after the blow to its pride inflicted by Vietnam, America's strength and optimism survived and were reflected in the triumphs of Star Wars.

As well as removing English pessimism from the mythic stories, Lucas adds another American characteristic. Like every hero, Frodo and Luke Skywalker must struggle with the forces of darkness. Skywalker must confront his worst enemy, only to find that he is trying to kill his father. It is a Freudian story, shaped for a psychoanalytic nation obsessed by the search within itself.

The British empire is gone, leaving a world dominated by the American dream. Until September this year, that American world was entirely sure of itself. This might have made audiences unreceptive to the Lord of the Rings films. They tell a story already told in some of the most successful films of all time, but instead of the open-ended future one finds in science fiction, and the American certainty that that future can be turned to the good, they offer only the pessimism of a vanished empire and its age.

The US may yet retain its optimism. Or it may be that a more uncertain world is now ready to receive films which remind us that power is transient and that victory is not an unambiguous good.

The Lord of the Rings: The Fellowship of the Ring (PG) is released nationwide on 19 December