After giving America the bomb, Robert Oppenheimer became the target of a political witch-hunt in the

In Washington in the spring of 1954, the scientist Isidor Rabi gave evidence at the McCarthy-style security hearings on Robert Oppenheimer, by then world-famous as the father of the atomic bomb. Rabi spoke of his friend's noble character, his scientific brilliance, his qualities of leadership and his loyalty to the United States and its institutions.

He also identified Oppenheimer's principal achievements, which included a leading role in pushing the US to the forefront of world physics, the creation of the bomb and some important nuclear policy initiatives since the war's end.

Finding the inquisitors still unsatisfied, a frustrated Rabi asked: "What more do you want? Mermaids?"

So dazzling is the story of Robert Oppenheimer, so rich in incident and enigma was the career that was destroyed in those shameful hearings, that in fact the appearance of the occasional mermaid in the narrative might not be that much of a surprise.

This was a man whose command of geology, when he was 12 years old, earned him an invitation to lecture before the New York Mineralogical Club. A man who in 1926 apparently attempted to murder the Nobel Prize-winning British physicist Patrick Blackett with a poisoned apple. A man who wrote scientific papers with Max Born and argued with Niels Bohr, but who also translated Hindu scripture from the Sanskrit and cited Baudelaire as his chief philosophical influence. And a man who, when he got home from work in the evening, could admire his own Van Gogh on the living-room wall.

A physical wreck whose every cough seemed likely to be his last, Oppenheimer liked nothing better than to sail straight into a thunderstorm or to ride the snowy mountain trails of New Mexico for days on end. He mixed the best cocktails and cooked the best steak dinners; no one could chair a committee as successfully as he could, and for a time his signature pork-pie hat was almost as readily recognised across America as Charlie Chaplin's bowler.

One day, as he prepared for those security hearings in 1954, working long hours with his lawyers (with the FBI listening to every word), he took a break, stepped outside and, bumping into a friend, asked for a bit of advice. Who was that friend? Albert Einstein.

These are just the adornments, the frills on the edges of his life. The central story of Robert Oppenheimer is, in many ways, the central story of the second half of the 20th century. He was the genius of the nuclear weapons age and also the walking, talking conscience of science and civilisation; most of the great questions surrounding him as a person were the greatest questions of that time.

He was born into an intellectual New York Jewish family and as a young man experienced the revolution in theoretical physics in the 1920s at first hand in Europe, before settling in California and building a world-class research centre there. Though he had no record as a manager, when war came he was chosen as the Manhattan Project's chief scientist and his inspirational leadership saw it through to success.

Peace found him a national hero and a powerful voice in Washington, but he was also increasingly anxious about the drift into Cold War. These qualms made him enemies, so his pre-war left-wing past was dredged up and, at those 1954 hearings, he was subjected to what one observer called a "dry crucifixion".

Should scientists make weapons of mass destruction? What role can scientists have in nuclear policy? Can politicians be trusted with these weapons? When is it right for free states to keep secrets and when should they be candid? How can you stop an arms race? Would a war fought with H-bombs be worth winning? All these momentous questions run through Robert Oppenheimer's eminently Shakespearean life.

And the best of it, at least from our perspective as readers, is that a vast amount of the debate and argument is on the record. Not just because he wrote a great deal, gave many interviews and sat on many committees where minutes were kept, but also because he must have been one of the most bugged people in America, a permanent target for a paranoid FBI. More money had been spent eavesdropping on his conversations, he once joked, than on the atomic bomb itself.

In American Prometheus, Kai Bird and Martin Sherman have produced a biography worthy of this extraordinary man, a work already rewarded with a Pulitzer Prize. Although more than 25 years in the making and 700 pages in length, it wears its scholarship lightly and whisks the reader through the story at thriller-like pace. We are given relatively little colour, reflection or background: characters are left to draw themselves by their words and deeds, none more so than Oppenheimer himself, a flawed hero even to his most ardent admirers.

Like some elaborate laboratory experiment, the intense events of a psychologically disturbed youth, of a personal intellectual flowering, of the bomb project, of the postwar politicking and of the final crisis are seen to test every player and every idea to the limit. And the story, for all that it is half a century old, becomes, in the hands of Bird and Sherman, a tale for today. It was fear that made possible the crime that was committed against Oppenheimer in 1954 - a hysterical, irrational fear of the Soviet Union.

In that climate, a democratic society was prepared to set aside civil liberties and common decency, to lose its head to the point where unbalanced fanatics who belonged on the fringe of civilised debate were given licence to tear down a great and loyal American.

Establishment Washington from President Dwight D Eisenhower down stood by as Oppenheimer's enemies created their own kangaroo court, with hand-picked judges, to try him. The prosecution had access to mountains of FBI material, much of it gathered illegally, while the defence did not.

Witnesses found themselves challenged over words spoken a dozen years earlier, by lawyers waving transcripts of bugged conversations. They were not allowed to see the transcripts to test whether they had been quoted accurately or in context, or to question whether the recordings had been made legally.

No element of risk was admitted. The press was squared. The president was squared. The judges were leant on. The defence was bugged. And because this was technically nothing more than a security clearance hearing, Oppenheimer's constitutional rights could be ignored.

Yet not a shred of credible evidence was ever produced to suggest that the man was disloyal, still less that he was a spy. The worst that could be proved against him was one or two lapses of judgement in dealings with left-wing friends in the early years of the war - piffling faults, as Isidor Rabi pointed out, when set beside his achievements.

So demented were his enemies that, even after what they described as his "unfrocking" - the official decision that he did indeed pose a risk to national security - they insisted he was on the brink of defecting to the Soviet Union and so must continue to be followed and bugged wherever he went.

The ghost of Guantanamo floats over these pages, and I suppose the good news is that the story has a redemptive conclusion. In the effort to justify themselves, Oppenheimer's enemies rushed the transcript of the hearings into print, only to find that the world read them and recoiled in horror.

In the 1960s, though he was in most respects a broken man, Oppenheimer was (to use an old communist term which seems fitting) rehabilitated, as John F Kennedy welcomed him to the White House and Lyndon B Johnson gave him a medal. America was sorry, and America was embarrassed.

But with Oppenheimer nothing is ever simple, and it would be a mistake to think that the story ended tidily and reassuringly there, as he drifted towards his death from cancer in 1967, at the age of just 62.

It is not something that could be proved, but Bird and Sherwin lay just enough of a trail to set you wondering: did Oppenheimer, the cosmic thinker capable of seeing a bigger picture than almost anyone, design his own martyrdom?

Did he go down in flames at least half deliberately, perhaps in the hope of bringing his country to its senses, of drawing the McCarthyist poison - or perhaps to make of himself a new Dreyfus, a new Galileo? Then again, you may find yourself thinking, perhaps that is just seeing mermaids.

Brian Cathcart's "The Fly in the Cathedral" is published by Penguin

Brian Cathcart is Director of Hacked Off. He tweets as @BrianCathcart.

This article first appeared in the 14 January 2008 issue of the New Statesman, Obama unmasked