Six months before he was murdered in his study in Mexico City, Leon Trotsky wrote: "I shall die a proletarian revolutionist, a Marxist, a dialectical materialist, and, consequently, an irreconcilable atheist. My faith in the communist future of mankind is no less ardent, indeed it is firmer today, than it was in the days of my youth."

There is something tragicomic in this confession of faith. Dialectical materialism, though it claimed to be based in science, was never more than superstitious gibberish. When he invoked the supposed science to bolster his failing political hopes, Trotsky was engaging in a type of magical thinking, using words as charms to ward off the terrors of history. At the same time - and this is the irresistibly comical element of Trotsky's career - he never ceased to regard himself as anything other than an uncompromising rationalist.

The comedy did not end with Trotsky's assassination, nor with the collapse of the Soviet Union. Those of his disciples who finally acknowledged that the communist future was not going to arrive did not give up on the dream of world revolution. Instead, an influential number of them found a surrogate for the failed communist experiment in the heartland of capitalism. America replaced the Soviet Union as the embodiment of human progress - and, it transpired, as the instigator of revolutionary wars. Christopher Hitchens pinpoints this ideological transmutation with pithy accuracy when, in this collection of essays, he notes that "the dynamic ex-Trotskyist Max Shachtman" was "much more the founder of neoconservatism than Leo Strauss". The neoconservatives who followed in Shachtman's wake never actually swung to the right of American politics. If they supported the Bush II administration, it was not because they had come to accept the status quo. It was because they saw in the Bush White House an opportunity to use US military power to promote a "democratic capitalist" version of Trotsky's permanent revolution.

Hitchens's account of the origins of neoconservatism has obvious parallels with his own political trajectory. He has always made it clear that, for him, the decision to invade Iraq was justified as the beginning of a revolutionary war. It is this continuing ideological mindset that accounts for many of the misjudgements he has made over the past decade. For Hitchens, that the Iraq war proved to be a disaster does not show the enterprise to have been a mistake - any more than the disastrous history of the former Soviet Union shows that the Bolshevik revolution (for which Hitchens continues to nurse a decidedly soft spot) was a mistake. In both cases, the human costs count for very little in the final analysis. What matters is the world-transforming revolutionary impulse that animated both experiments.

There will be some on the left who admire this heroic indifference to consequences. Better persist in attempting the impossible, they will say, than embrace hopeless realism. The trouble is that, in politics, the pursuit of the impossible so often unhinges the mind, to the point of blocking any reliable perception of the course of events.

When Trotsky urged his American supporters to mount a campaign against joining Britain in an imperialist war in 1940, at a time when the Nazis had snuffed out practically all that remained of democracy in Europe, this was more than a strategic miscalculation. It was manifestly delusional. The same must be said of Hitchens's assertion in 2007, reprinted here: "The world now faces a challenge from a barbarism that is no less menacing than its three predecessors - and may be even more so."

There are some who represent Hitchens as a contrarian or provocateur, without convictions. They are wrong. What sort of provocateur would write that "Bin Ladenism" is more dangerous than German Wilhelmine imperialism, the Nazi-Fascist axis and international communism? Such a patently absurd claim could only be made by one who deeply believes it to be true.

Leave aside the grotesque disproportion in lumping the Kaiser's Germany in with mid-20th-century totalitarianism. What is wholly fantastical is putting Osama Bin Laden's gang in the same category as Nazi Germany and the Soviet Union - two extremely powerful states with vast industrial and military resources, the first coming close to conquering all of Europe, the second annexing Europe's eastern half and the Baltic states while imposing itself throughout central Asia. In passing over these undeniable facts, Hitchens is not playing the role of intellectual gadfly. He is showing himself to be a believer who - like Trotsky - blanks out reality when it fails to accord with his faith.

That Hitchens has the mind of a believer has not been sufficiently appreciated. His critics usually fasten on secondary features of his work, quite often those that make reading him so enjoyable. It may be true that he is a bit of a name-dropper, and yet the conversations he recounts are never reported for effect; they are absorbingly interesting in their own right. It is true that there is something indescribably English in his style of writing - though why this should be a fault is not clear.

Reading Hitchens, one cannot help thinking of the combative and unsparing wit on display in Claud Cockburn's journalism; but, by any reasonable assessment, Hitchens is a far more substantial figure. To fasten on his role as a celebrity journalist (as many of his critics have done) is to underestimate his achievements, because, when he leaves behind the certainties of ideology, he is an incomparable truth-teller.

Perhaps these critics have forgotten that many true believers are those most closely acquainted with doubt. That paradox is evident in the hundred and more short essays collected in Arguably. There are many in which it is Hitchens the impassioned sceptic that is on show. "The Vietnam Syndrome", a masterpiece of lapidary finality published in Vanity Fair in 2006, concludes with the observation that many of the war's casualties are victims of Agent Orange yet to be born - "and if that reflection doesn't shake you, then my words have been feeble and not even the photographs will do".

In an incisive analysis, Hitchens defends Winston Churchill from revisionists who think that peace was achievable and desirable in 1940. In a moving essay on the diaries of Victor Klemperer, Hitchens brings out a trait of the Nazis that is frequently overlooked - the calculated sadism that led them to ban the ownership of pets by non-Aryans, and compelled the Klemperers to have theirs put down. A prescient examination of the euro considers what might be the political consequences in the event that the brittle and divisive construction falls apart. These are just a few of the exemplary essays that one could cite. Dozens more of the pieces collected here are just as good.

Coming from one of the greatest living writers of English prose, Arguably is the testament of a prodigiously gifted mind. To say that, during the past three decades, the world would have been poorer, duller and altogether a smaller place without Hitchens and his writings would be to utter a cliché of the kind he despises. It would also be true.

John Gray is the NS's lead book reviewer.
His latest book is "The Immortalization Commission: Science and the Strange Quest to Cheat Death" (Allen Lane, £18.99)