''I never intended to challenge Gordon''

Caricatured as an über-Blairite and criticised for the leadership challenge that never was, David Mi

One afternoon in January at a private lunch at the British high commissioner’s residence in Delhi, a fine, white-painted colonial-era house on the flat roof of which were perched several watchful Gurkhas, David Miliband, the British Foreign Secretary, sat at a long, varnished table eating salmon mousse and salad leaves while he took questions from veteran Indian military leaders and retired ambassadors. The lunch began congenially but soon curdled into disagreement; the atmosphere became tetchy. The Indians had been outraged by the November attacks by Pakistan-based Islamist militants who, arriving in Mumbai by boat, had launched ruthlessly violent commando-style raids on shining symbols of Indian affluence and modernity. The Indians were eager to know whether the British government was prepared to condemn Pakistan for what they believed was its complicity in the attacks.

Miliband repeated what he would say throughout what turned out to be a gruelling and contentious four-day visit to India. He had seen evidence, he said, that the attacks had come from within Pakistan but that they were not directed by the Pakistani state, and that those conspirators who had been arrested in Pakistan should be tried there rather than extradited to India, as the Indian government wanted. His comments were received with irritation. The mood was set for a lunch of cross-cultural misunderstanding: Miliband took questions and answered them deftly, sometimes confrontationally, as if he were preparing for a session at the Despatch Box, while his elderly interlocutors repeatedly referred to him as "young man" (he is 43), and one, at that, as some of them said afterwards, who did not fully understand the history, complexities and tensions of the India-Pakistan conflict.

"The young man wasn't listening to anything I said," one retired high commissioner said to me at the end of the lunch. "I didn't register on him at all."

Later, as we travelled by chartered jet from Delhi to a remote airfield in rural Uttar Pradesh, where we were to be met by Rahul Gandhi and taken on a tour of villages in his Amethi constituency, I told Miliband about my conversations with some of the disgruntled Indians and what they had said. "But I was listening very carefully," he protested. "They did register on me. What they were saying dramatised for me the chasm [between India and Pakistan]. I understand their hurt and anguish - which is why I'm here on this trip, to show solidarity with India."

Miliband may have come in peace and solidarity but once he had returned to London he found himself being widely disparaged in the Indian media, disparaged for his tone, attitude, sensi­bility and hauteur. He had not shown enough "deference" to India's foreign minister, Pranab Mukherjee, who is 73, and Prime Minister Manmohan Singh, who is 76. He had addressed them by their first names, it was reported, while they referred to him in return as "Your Excellency" or "Mr Miliband". His visit was an example of "disaster diplomacy", said the opposition BJP, who were angered that Miliband had spent so much time with Rahul Gandhi, the latest member of the Gandhi-Nehru dynasty to emerge as a contender to lead the Congress Party and thus become prime minister. Gandhi had invited Miliband to see "the other India", the India of the villages where more than 700 million people live and struggle daily with poverty.

A grave problem for him is that his high intelligence and sarcastic wit can be mistaken for arrogance

It was reported that the Indian government had sent a letter of complaint about the British Foreign Secretary to Gordon Brown. (This is not the case.) "Clearly, no self-respecting national leader is going to invite him back to their country," wrote one journalist. "Not that he's winning the popularity sweepstakes in Britain - there, Millipede, as some sections of the press call him, is regarded as a covetous courtier who wants to unseat UK Prime Minister Gordon Brown. However, what he's doing abroad is what Whitehall should be worried about . . . One would expect that a British leader visiting India, weeks after the Mumbai terror attacks, would be properly briefed by their Foreign Office about India's sensitivities."

As I discovered, Miliband had been briefed rigorously. He held India's senior politicians in the highest respect. "What a fantastic country this is, isn't it?" he said to me one evening at the high commissioner's residence. He was sitting slumped in a chair after a long day of meetings. He was obviously tired yet still spoke with a kind of boyish wonder and enthusiasm. "All these Indian leaders have PhDs," he continued, tossing a cricket ball from hand to hand and picking at its seam. "What they've achieved here is just fantastic. But there's such deep anguish about the attacks - such anguish, anguish, anguish."

Miliband is a politician of unusual integrity - and that can be part of his trouble. He believes that, in diplomacy, "you do not say one thing in private and another in public. You do not say one thing to one group and something else to another." This is why throughout his trip to India he refused to deviate from the British government's position on Pakistan, whether at private bilateral discussions or at public meetings and press conferences. "In our first ten years in office we didn't do a good enough job explaining the motivations for our policies," he told me. "This was the case with the most controversial things we did, with Iraq being an example: people weren't clear about our motivations. You've got to get to the stage where people can disagree with your policy but understand your motivation. If people disagree with your motivations you're in a very divergent position."

Miliband's motivations were transparent, at least to me, in India; it is true that he said nothing in public that he did not say in private. And yet his visit was perceived as disastrous. Why? The answer, I think, is that if Miliband has a flaw it is that he lacks what the psychologist Robert Sternberg calls "practical intelligence". To Sternberg, practical intelligence is "knowing what to say to whom, knowing when to say it, and knowing how to say it for maximum effect".

The American writer Malcolm Gladwell, who has popularised Sternberg's ideas, has offered his own definition of practical intelligence. It is procedural, he writes in his latest book, Outliers, "it is about knowing how to do something without necessarily knowing why you know it or being able to explain it . . . It's knowledge that helps you read situations correctly and get what you want. And, critically, it is a kind of intelligence separate from the sort of analytical ability measured by IQ."

Not only did politicians such as Tony Blair and Bill Clinton have boundless imaginative empathy, they had superior practical intelligence: they invariably knew what to say and when to say it, to maximum effect. They could assess the logic of a situation and adapt accordingly. At the high commissioner's lunch and at other times during his visit to India, Miliband displayed no such skill. He made little attempt to indulge or charm or defer to the assembled guests. He said what he had to say, briskly and with donnish confidence. His analytic intelligence was impressive: he was the master of his brief, he was superbly articulate, he was dominant. But his practical intelligence was less impressive: he showed his irritation too easily, for instance.

I told him that this was so on the plane to Uttar Pradesh. "It's a good point," he said, nodding. "Look, you learn every day in this job . . . you've got to try and take that forward. I know words matter in diplomacy."

"There's a comparison to be drawn between David Miliband and David Owen," says a former colleague of Miliband's. "Owen was younger than David when he became foreign secretary and had great impact at the Foreign Office. Owen had a political self-confidence of the kind that David doesn't yet have. David needs to relax more. He's intelligent but still a little insecure - that's what makes him appear so haughty, at times. He's not naturally easy with people. He can be awkward and fidgety. Over time all that should get better."

David Miliband remains perhaps the most misunderstood senior politician in Britain. In person, he is hugely engaging: alert, intelligent, witty, candid, a little cocky. He listens hard. He remembers what you tell him. He relishes intellectual confrontation and engagement. He jousts and teases. On more than one occasion he introduced me as the “editor of the New Statesman”, adding, “It used to be a great magazine.” When, late at night, at the end of the long, exhausting tour of the Amethi villages, and shortly after he had spoken to Shimon Peres on the phone about the Israeli assault on Gaza, he was told by one of his aides that Osama Bin Laden had issued a statement, he quipped: “What, about my visit?” before returning to his plate of dhal and rice.

Miliband has an intense, restless curiosity and ferocious energy. He is very tall, with long, thin legs, and he walks quickly. There is nothing flamboyant about him. He wears simple, unpretentious suits, cut just too short in the leg, and plain shirts and ties. He has thick, closely cropped wiry hair, a bespoke style that is somewhere between a grown-out crew cut and what the Americans call a buzz cut. "With that hair, he looks like Action Man but without the scar," said one of his bodyguards.

His movements can be jerky, abrupt. At one point during one of our conversations, he suddenly used his hands to lift one long leg to fold it awkwardly across the other, like a contortionist preparing an elaborate move. He shows his boredom easily; when he is in a meeting which does not interest him or doing an interview that irritates him, his dark, almost black eyes begin restlessly to roam the room, and an expression of peculiar blankness settles on his face. "Get me out of here," he seems to be saying.

“I said he [Brown] was the right man last year and the right man this year and he will be the right man next year”

A grave problem for him is that his high intelligence, and sarcastic wit, can be mistaken for arrogance. "The trouble with David Miliband is that he is too grand," I was told by someone who is close to the Prime Minister and works at No 10. "He hasn't got that common touch you need as a Labour MP, especially a Labour MP who wants to lead the party. He doesn't get out there around the country enough, not like Ed Balls and James Purnell do. He doesn't mix with the MPs enough. That's why he was so battered last summer when he stepped forward to challenge Gordon."

This was a reference to the events of late July and early August 2008 which followed Labour's defeat to the Scottish National Party in the Glasgow East by-election when, with a demoralised Gordon Brown on holiday in Suffolk, with Labour as many as 25 points behind in the polls, and with MPs insurgent and openly discussing a leadership challenge, Miliband made his move. First, he published an article in the Guardian on 29 July - which, in retrospect, reads like the draft of a manifesto for a fourth Labour term: "With hindsight we should have got on with reforming the NHS sooner. We needed better planning for how to win the peace in Iraq . . . We should have devolved more power away from Whitehall and Westminster." The absent presence from the article was the name Gordon Brown.

The night before the article was published, Miliband's aides, perhaps operating freelance, were ringing lobby journalists to alert them to it. "If you want courage read the Guardiantomorrow," they said. Why courage? Well, some of Blair's closest supporters had wanted Miliband to stand against Brown for the leadership; they did not want the chancellor to become prime minister unopposed. Some viewed Miliband as the young Blairite pretender, a New Labour loyalist, a moderniser, a true believer, Tony's choice - Blair had once called him the "Wayne Rooney" of the cabinet and had spoken of Miliband, head of the Downing Street Policy Unit during Blair's first term, as being the anointed heir. "As head of the unit David was cautious rather than daring and creative," Roger Liddle, another former policy adviser to Blair, recalls. "He was 75 per cent where Tony was, not 125, if you see what I mean. He wasn't a Blairite outrider."

In the event, Miliband chose not to stand against Brown, not because he lacked courage but because he recognised the timing was wrong. His reward was to replace Margaret Beckett as Foreign Secretary. It was the summer of 2007 and he was, at 41, one of the youngest occupants of the position (David Owen was 38 when he became foreign secretary in 1977, as was Anthony Eden in 1935).

Two days after the Guardian article Miliband was interviewed on Radio 2's Jeremy Vine Show, taking questions from listeners. For 24 hours after that, he and his aides were ubiquitous. If this wasn't a leadership bid, it certainly had the feel of something co-ordinated. The BBC political editor, Nick Robinson, was prompted to say of Mili-band: "This is a man testing the waters for a leadership bid and a man simply unprepared to come to the defence of a beleaguered prime minister."

Towards the end of my week in India, on a flight from Uttar Pradesh to Mumbai, I asked Miliband to reflect on last summer and the events that followed. Why - if, as he says, this was not a tilt at the leadership - was it perceived as being so? "Because British politics is obsessed with personalities," he said, drinking coffee and tugging at a sealed sachet. It suddenly split, spilling white powdery beads across the table in front of us and over his dark suit trousers. "I thought that contained mints," he said, dusting himself down, "but it's sweeteners."

His concentration restored, he said: "Listen, I've always said that Gordon will lead us into the next election. I said he was the right man last year and the right man this year and he will be the right man next year."

So you can say, categorically, that it was not a leadership bid?

"That's right. It was not a leadership bid. Categorically it was not."

What was it all about, then?

"People were very worried. We'd just lost a by-election, and the whole party wanted the government to pull together and pull through. What I did was about cabinet ministers fulfilling their responsibility to speak for the whole government and to speak for the party. We had spent the weekend at the national policy forum [from 25 to 27 July at Warwick University] and people were very, very despairing. And I felt just to go away on holiday and hope it would just get better was wrong. In the end, you have a responsibility to speak up. That's what happened. In the end, this is about a strong Labour Party, about a party that manages to slap off its century-long addiction to being a party of good opposition, not a party of engaged, sensible, reforming government. We have established a bridgehead in the last 11 years that we'd never established before. What people are saying, what they were saying, is 'Work hard or we're going to throw it all away'."

Speculation about a Miliband leadership challenge simmered throughout the late summer months and on into the conference season. The briefings and plotting were febrile at the Labour party conference in Manchester at the end of September; in the hotels and bars the talk was about not if there would be a challenge to Brown, but when.

The Manchester conference is widely considered to have been hugely damaging for Miliband. The briefing against him from some of those close to the Prime Minister was as pernicious as it was unceasing. Gordon Brown, in a strong signature speech, pronounced that this was "no time for a novice", a remark that was perceived as a dual challenge to the two Davids, Cameron and Miliband. Miliband's own speech was coolly received. And one morning, to his profound regret, he was photographed holding a banana - his self-described breakfast - outside the conference hall.

With the global financial crisis deepening by the day, Labour MPs began to pull back from the idea of a leadership challenge. The movement for Miliband lost momentum. It was as if he had presented himself to the party and the party had turned away. Or, perhaps more accurately, the party had reacted with indifference. Then, in an astounding coup, and in the week after the end of the conference, the über-Blairite Peter Mandelson returned to cabinet as Business Secretary.

"Anyone who tells you they weren't surprised by that is lying," Miliband says now. "I wasn't consulted about it, I wasn't told in advance. I can't remember where I was when I heard. He's a big beast, a talent that had been lost from the British government, from British politics. What are his attributes? Weight. Breadth. Coolness."

I was told by one of Miliband's aides that the Foreign Secretary left Manchester feeling he had had a good conference - another failure of practical intelligence. "David himself was oblivious to all the briefings against him," I heard. "The speech went well. The Fabian event at the fringe went well. The discussions were constructive. What we didn't know was what was being said and going on behind our backs. We don't operate any systems of leaks and briefings - it's not the way we do politics. Are we damaged because of it? I don't know. Maybe."

The overall effect of this period of ambiguity, conspiracy and paranoia has been to force Miliband to withdraw, chastened, from the domestic scene, as Alastair Campbell, who knows him well, acknowledges. "It's a pity that the Foreign Secretary's job has taken him away from domestic politics somewhat," Campbell told me. "David has a young family; there's a lot of travelling. That job can take you out of the domestic policy swim. I think he has been missed on the domestic scene. He was excellent at Education and at the Environment and I would like to hear him more on these issues, too.

"It wasn't a leadership bid last summer. In politics you have to get noticed. In an ideal world his actions would have been part of a larger strategy, agreed by the top. Gordon needs more big figures out there fighting for him, the government and the party. Those who put the boot into David last summer have not done anyone any favours, Gordon included. Obviously, being Foreign Secretary is a busy, difficult job, but he should still be making contributions publicly on domestic policy and political strategy, in the context of an agreed strategy. He shouldn't care or think what people might say about his motivations. He's got to ignore all that stuff. Politics is a team game, particularly in our system, and Gordon has been taking too much of the load. Not just David, but the other ministers, need to be more visible, need to connect more with the public."

Campbell is right: since the autumn Miliband has been preoccupied by a succession of international crises - the war in Georgia, India-Pakistan, the catastrophe in Gaza - and has made no significant contribution at home, nor given any major speech on domestic policy. When I first began talking to him, over lunch before Christmas, he expressed the hope that he would be able to have a quiet end of year with his wife, Louise Shackelton, a violinist with the London Symphony Orchestra, and their two young sons, Isaac and Jacob. "Last Christmas [2007]," he said, "there was the assassination of Benazir Bhutto and the trouble in Kenya. I was on the phone nearly the whole time."

As we parted, I wished him well - and a quiet Christmas. He mentioned in passing that he was fearful about Gaza and about the fragility of the precarious peace between Hamas and Israel. When I next met him on the overnight flight from London to Delhi, he had only just returned from New York where, at the United Nations, a UK-led resolution on Gaza had, one of his aides told me, "come within minutes of being passed".

Miliband seemed exhausted - he had given a statement on Gaza in the House that afternoon - but was soon revived, as restless and animated as usual. He reflected with frustration on a brief interview he had given on BBC2's Newsnight during which Jeremy Paxman had called on him to condemn the Israeli action in Gaza. "I should have said that we always condemn loss of life," he said.

In spite of his tiredness, he was friendly and he wanted to talk, to hear what I thought about the catastrophic assault on Gaza and about the Mumbai attacks. If he disagreed, he said so, or demanded greater precision of explanation. If I said something he liked, he said "That's interesting" or "That's clever" - words and phrases which, during later meetings in India, I would hear him use again and again, verbal tics that served as punctuation points, breakers in the torrential stream of words and ideas.

On foreign policy, Miliband's position has philosophical coherence in an age of globalisation. He is a committed European and multilateralist; he expresses no enthusiasm for British adventurism in Iraq and makes no attempt at retrospective justification. "Foreign policy is inseparable from domestic policy now," he says. "Is terrorism foreign policy or domestic policy? It's both. It's the same with crime, with the economy, climate change."

He mentioned on several occasions that "values matter because people need to know your motivations". So what, I wondered, was the motivation for the war in Iraq? "We went to war to defend UN security resolutions and, secondarily, to defend human rights."

He made no mention of weapons of mass destruction.

"The multilateral system broke down [over Iraq]. Yet one of the big differences between Labour and the Tories is that the Tories say international institutions are a threat, not just to British sovereignty, but to British freedom. We say the weakness of international institutions is the great threat to British freedom, because you need those institutions to defend freedom."

Throughout his trip to India, whether he was working late at night to refine a revisionist speech on the war on terror that he gave at the Taj Mahal Palace hotel in Mumbai (“Is this quite the right sentence?”) or preparing, at the invitation of Rahul Gandhi, to spend the night in a mud hut in rural Amethi (“If I eat the food here, I’ll certainly be needing en suite facilities”), he never stopped agonising over Gaza. He took calls from his counterparts in other countries. He was continually expressing sorrow at the increasing numbers of Gaza’s dead. On more than one occasion he expressed despair that he would “never see the creation of a Palestinian state in my lifetime”.

When pressed on this, he said: "Every year this thing doesn't get done, it gets harder to get done. It's clear what the endgame is: security for Israel and normalisation with the whole of the Arab world. The Palestinians can't give Israel security. It takes the whole Arab world to do that. What Israel can give the Palestinians is land. They have to give the 1967 borders plus or minus land swaps."

Hamas has said it demands a return to pre-1967 borders?

"Hamas has said many things. But the international consensus is deafening. It's a fair settlement on refugees; it's a sui generis resolution to the issue of Jerusalem, with Jerusalem becoming the capital of both. I can't admit to myself that the creation of a Palestinian state won't happen. What I know is that with each passing year it gets more and more difficult to happen, not least because there is more and more blood shed, generation upon generation."

Would Britain speak to Hamas?

"The Arab League has nominated Egypt to speak to Hamas, and that seems to me to be reasonable. The Syrians and the Turks speak to Hamas, and we speak to the Syrians and" - he paused - "Anyway, there's no shortage of people speaking to Hamas."

What does he think of some of those on the left who have compared the plight of Palestinians in Gaza to that of Jews in the Warsaw Ghetto? "That's not just disproportionate, it's wrong. The Holocaust was a unique historical event. It was the attempt to wipe out an entire race. You don't need to compare it to the Holocaust to dramatise or engage with what happened in Gaza; 1,200 people lost their lives."

One evening at the high commissioner’s residence in Delhi, I asked Miliband, shortly after he had given an interview to an Indian current affairs show, if there was a distinctive post-Blair foreign policy. “Well,” he said, as we waited for a group of Indian politicians to arrive for dinner (in fact, only two of the expected nine turned up), “it’s our job to define post-Blair foreign policy. Things move on. Tony himself has said that he didn’t pay enough attention to China and India. He used to talk about Britain as a bridge between Europe and America. I think that’s inadequate, because there’s a big world out there. That’s why I talk about Britain being a global hub. That’s not meant to be vainglorious or empire-building; it’s just to recognise that Britain can be influential and strong and engage with China precisely because we are strong in Europe. That’s a difference [from Blair]. The challenges and threats are clear to us. They are about insecurity, terrorism, about unsustainability of climate and the weakness of international institutions. There’s the relative decline of American power and the rise of economic power elsewhere. The Americans can’t bring the world to heel on their own.”

How damaged has Britain been by its close association with the Bush administration, with the shameful administration of unilateral declarations of war, Abu Ghraib, Guantanamo Bay, of extraordinary rendition, of torture? "Look, we are confident that we will have a vibrant relationship with Obama. We have shared values and shared priorities. He ran the most remarkable campaign and represents perhaps the most remarkable political rise in American history. It's the most remarkable achievement, done with huge grace, huge calm, huge karma and huge determination.

"I don't think we're too close [to the United States]. We stand on our own two feet. People can disagree with aspects of our foreign policy but they will still want Britain strongly to play an important role, whether in DRC, in Europe, in the Middle East, in south Asia. You're never going to get me to say that a Labour government has damaged the country. You're never going to get me to say that. People were disappointed by our stand [on Iraq] but they still have huge respect for the UK. Let's not have rose-tinted spectacles about it [sic], but they still accept Britain has an important role to play."

As for US-sponsored torture, he is unequivocal. "I abhor anything that constitutes torture. Water-boarding, it's perfectly clear to me it is torture. I never supported extraordinary rendition to torture, always said that Guantanamo should be closed. There is no clash of ideals and prag­matism there. Are your principles tested in this job? Of course they are. You have to weigh things carefully. The strength of your ideals makes you hard-headed."

In foreign affairs, as in politics in general, there is always an ambiguous space between rhetoric and reality, between professed values or ideals and action, between motivation and policy. Miliband professes to be an idealist but his ideals are continuously being challenged, most recently in the case of Binyam Mohamed, the British resident from Pakistan who was kidnapped - or rendered, to use the preferred euphemism - and taken first to Morocco and then Afghanistan, where he was tortured, followed by his present incarceration at Guantanamo.

British judges ruled that journalists cannot have access to reports explaining how exactly Mohamed was tortured, a decision that Mili­band defended. "It is US information and it is for the US to decide when to publish their information," he said.

"Look," he says now, "I abhor the practice of torture. We take our own responsibilities extremely seriously. The Mohamed case is an example of where our pragmatism has not violated our principles. We have stuck up for our principles by saying that he should have the documents for defence. We have stuck up for our principles by saying we should get him back. But there's been this elision between that and somehow saying we're denying justice. We're not."

I returned him to Blair's conference speech of 2001, delivered in the stunned, time-slowed aftermath of the 11 September attacks, as the world waited in fear for the American response. At his most ardently messianic, Blair had spoken of this being a "moment to seize. The kaleidoscope has been shaken. The pieces are in flux. Soon they will settle again. Before they do, let us reorder the world around us . . ."

We know how Blair and Bush chose to reorder the world - through illegal declarations of war. Does Miliband regret Blair's liberal utopianism, his desire to remake the world? "It wasn't liberal utopianism. No, not utopianism. It was harsh realism. He was saying that these events have got real causes and those real causes involve very, very difficult issues in which history overwhelms if not rationality, then compromise. To that extent, it was quite realistic."

But your approach is different from Blair's? You're a multilateralist and he, ultimately, was a unilateralist?

"That's for other people to say. Now we are looking at a situation and trying to get ahead of it, ahead of the curve, as Gordon is doing by leading on the economic recovery - he's trying to get ahead of the curve. On climate change we are trying to catch up with the curve. The Gaza explosion is a symptom of failure. With a lot of foreign policy it's about making good problems that haven't been resolved in the past. Gaza is a symptom of that failure. In this job you're trying to mend things while getting ahead in the strategic sense, trying to anticipate where the threats and problems are coming from."

As for Afghanistan, President Obama may have authorised the deployment of as many as 17,000 additional US troops - "the security needs are urgent," he says - but Miliband is more circumspect. "The need to send more British troops to Afghanistan is not proven. We'll have to see. It depends on strategy. If Americans bring the issues of Pakistan and Afghanistan together, we can help them work very closely on that. How long will we be there? It depends on our progress in training up parts of the Afghan national army. The development challenge is a much longer term one."

One recent rainy morning in London I visited Miliband at his magnificent office in Whitehall. He was standing at the window, beside a bust of Lord Grey, looking out at the blue winter haze, the rain falling. He was as open and energetic as before, and he reflected once more on Labour’s summer of madness in 2008 and his own contribution to it. “What happened back then was all about fulfilling the responsibilities of being a member of the government, not just a leader of a department. Gordon leads the vision, he leads the government. The rest of us are in a supporting role. We make the government actions achievable and we have to show that we learn from our, er, less stellar achievements.”

He paused as tea was served. "We have to contribute to a sense of energy, drive, ideas. What's interesting about the current [economic] situation is that no one could sit around a table and say, 'Well, I really think those Tories have sketched out a clear shape and understanding of the modern world, that they've defined a policy agenda that's got rigour and drive at its heart.' I can't see the Tory vision of Britain in 2015 or 2220. There's a reason for that - there's a hollow centre at the core of the modern Tory party. Just as Labour re-found its purpose in 2008, the Tories were exposed as having no purpose at all."

Have you been impressed by the Prime Minister's recovery, by his overall performance?

"Gordon has shown outstanding leadership on the economic crisis. He's made the right decisions in not seeking to follow fashion but to do the right thing - on recapitalisation of the banks, on public spending, on further lending, on saying that the crisis is global. He's got enormous determination, real grasp of detail, the ability to stand back and look at the whole scene, and also he's got a real sense of purpose. He's doing this because he's protecting the British people while at the same time recognising that he's got to protect the climate-change agenda and the development agenda."

You said that, in 2008, Labour had re-found its purpose. But you can't re-find something unless it has first been lost?

"OK. In 2008, the party defined its purpose - I think that's better. In 2009 we've got to show strength, unity, resilience and determination. In 2010, Gordon will be able to put a clear and strong vision to the country."

He went on: "What Gordon has done, in part because of the economic crisis, is to put fairness - whether in terms of the top tax rate or in protecting the child poverty strategy - at the heart of the political, social and economic agenda. That's why he's got narrative rhythm; it's why he's got a clear sense of priority. It's always hard when you are in a stage of deep administrative difficulty and before the decisions you have taken have had their full impact. People always want you to change course. You mentioned [the French president] Sarkozy's criticism of our VAT cut, but this is one of those things where you don't follow fashion but do the right thing. You've got to protect people in a downturn. A Labour recession is different from a Tory recession because you try to protect people properly and also prepare for the upturn. We're a government of politicians, not an administration of technocrats. In the hundreds of decisions you make in our overly centralised country that red thread gets lost. Part of our job is to make sure that red thread isn't lost."

I asked him what he meant. Was the red thread of which he spoke the red thread of socialism - the red thread of his father's politics?

“He’s got exceptional brainpower. He’s got a remarkable capacity to see to the core of an issue”

"The core of the New Labour insight was to put social justice at the heart of the British economy. What I mean, then, is the red thread of Labour, of what we represent. People want to vote for politics, not for an administration. They want good administration, and competent administration, but they vote for politics.

"In domestic and international policy things have changed in fundamental ways. There's a crisis there but also an opportunity there. I passionately believe that you cannot solve the problems of the modern world, international or domestic, without progressive values. That doesn't mean progressives will always win elections. It means you will be in with a shout. You are in with a shout if you translate those values on fairness, co-operation, mutual responsibility into tangible, hard, crunchy politics.

"The point is, if you're a 12-year incumbent government you have to work quadruply hard to be the insurgent, not the Establishment . . . You've got to be doubly self-critical, got to be doubly engaged in making sure that you don't become the complacent Establishment."

He does not accept that there is even a covert, low-level leadership campaign going on in the Labour Party, with cabinet rivals such as Ed Balls and Harriet Harman jostling for position to be the ultimate successor to Brown. "The only campaign going on is to secure the re-election of a Labour government. It's good that we've got young people coming through the party, but what motivates me is getting the best possible result for Labour at the next general election. We've all got to fight like hell to win every seat."

His brother, Ed, is obviously one of those young talents?

"Yes. I'm very proud of him. He's very talented."

It's said that he's to the left of you?

"Oh, let people make their own judgement on that," says Miliband. "He's much better than me in all sorts of respects and he's doing a job in a very important area."

In his Guardian article of July 2008, he wrote of the need for "a restlessness for change", for New Labour to continue reforming, challenging, remaking. Miliband is Jewish, but not a believer. Reflecting to me on his identity, he said: "I'm conscious of my roots, of the journey my family has made. I'm proud of it" - his voice, quieter now, quickened - "very proud. My wife isn't Jewish. I'm not active in the community, but I'd hate it if it died out - not the Jewish faith in general, but a sense of it in my own family. My children never met my father but I want them to have a sense of it, yes."

As an atheist, he is committed to practical solutions in this world, in the here and now, and he is in a hurry, never complacent. The son of the Marxist academic Ralph Miliband and a mother, Marion Kozak, who was a history teacher and later a care worker who studied for a PhD, he grew up with the expectation that the "class struggle was there to be fought. Yet we were middle-class. The 1970s - I was ten in 1975 - were a bad decade in all sorts of ways but the middle class had comfortable assumptions about the prospects for its children. The middle class was smaller then; it was a much less competitive Britain, less meritocratic. The expectation was that people would land on their feet. Now there are more people in the middle class, which is a good thing, but there is also more competition, and people are more worried. There is more insecurity and anxiety of a kind that I wasn't brought up with."

Miliband is well liked by his staff at the Foreign Office. Unlike his predecessor Margaret Beckett, he is spoken of, in that slightly condescending way of diplomats, as "a natural foreign secretary". The office has been reinvigorated. "Tony had become such a dominant figure in foreign policy that the FO had lost a lot of confidence," Miliband says. "We now have a much more in­tegrated operation. We've developed a strategy and we've stuck to it. We have a clear sense of what we're about."

"We like to think we are cleverer and sharper than other civil servants," one of his aides told me. "More than that: you've got intimidating Victorian buildings here, which have not been open-planned and gutted as has happened elsewhere in Whitehall; you've got embassies abroad; you've got the outdated trappings of power, our inheritance as a former imperial power. But David has not been intimidated by any of this. He's got exceptional brainpower. He's got a remarkable capacity to see to the core of an issue, to take a complex problem and to analyse it and disinter its key elements and put them back together in the correct order that leads to a conclusion of what we want to try to achieve.

"Some foreign secretaries concentrate on the tactics, on getting through the next meeting and the one after that. David focuses very much more on identifying a strategic goal and working on a path to get to it. He's challenged the whole office. He has tested our assumptions, tested our cultural predispositions to certain positions in international relations. He has a genuine openness to intellectual debate, to finding the right answer. He's set a high bar."

Indeed, he has. However, there is still the problem of his practical intelligence, or lack thereof, of his inability sometimes to read the logic of the situation, to seize on it and respond effectively. There is a sense, too, that Miliband is at his best when he is free to be most himself: the intellectual working on strategy, finding a way forward, debating and analysing, showing the natural attributes of the policy wonk. So can he connect with the wider electorate, let alone with fellow MPs and those in his own party who accuse him of being too remote or, more disparagingly, as not quite of this world?

Miliband spoke repeatedly to me of the need for one's motivations to be understood, but last summer very few people, if any, understood his motivations. It is possible that he himself did not understand the reason he chose to act as he did when he did: failing to praise and support Gordon Brown, the Prime Minister, when the need was most urgent and Brown's very survival was most threatened. There are whispers from Downing Street that Miliband was not at last month's World Economic Forum in Davos, Switzerland, because "he is being marginalised again by Gordon".

I don't believe that. I think David Miliband was cautioned and wounded by the events of last summer and, more recently, by the blowback from his trip to India, a trip caricatured in the Indian media as a debacle. He has chosen, for now, to withdraw, to commit himself fully to the relentless churn of his work. "All this is very hard on my wife, very hard on the kids. I don't think you can get a work-life balance, but when you have your family time you must not have your BlackBerry with you. Don't text when you should be playing with your kids. There's a guy with a gun standing outside the door of our house. I wonder sometimes how that must affect the children."

Miliband is weary of the culture of disparagement that surrounds him - the sniping and the malign briefings. "I believe that we have a gotcha culture," he said. "In the end, you've got to believe that people will judge you on how you do the job. The privilege of being Foreign Secretary is an extremely good way of venting any frustration you might have. What I mean is, you focus on the job. Last Thursday I saw Dick Holbrooke for breakfast; I saw the Luxembourg foreign minister; I saw General Dayton, the Middle East peace co-ordinator; we put out a statement; I debated about Pakistan and Afghanistan in the House; I had tea with General Petraeus; I put my children to bed, and then I had dinner with President Abbas. It's a privilege to have that kind of day."

David Miliband: the CV

15 July 1965 Born in London, eldest son of Marion Kozak and Ralph Miliband, Marxist theorist and co-founder of the New Left Review. His younger brother Ed is Secretary of State for Energy and Climate Change. He and his wife, Louise Shackelton, a violinist in the London Symphony Orchestra, have two sons

Educated at Haverstock Comprehensive and Corpus Christi College, Oxford, where he studies PPE; later studies at MIT as a Kennedy Scholar

1989 Becomes a policy analyst for IPPR

1992 Appointed secretary of IPPR's Commission on Social Justice by the Labour Party leader, John Smith

1994 Head of policy for Tony Blair; is a major contributor to the Labour Party's 1997 manifesto. Becomes head of Prime Minister Blair's policy unit after election

June 2001 Elected MP for the safe Labour seat of South Shields

June 2002 Becomes schools minister at the age of 36

December 2004 Replaces Ruth Kelly as minister for the Cabinet Office

May 2005 Appointed to newly created cabinet position of minister of state for communities and local government

May 2006 Appointed secretary of state for environment, food and rural affairs in Blair's most far-reaching cabinet reshuffle

June 2007 Appointed Foreign Secretary. At the age of 41, he is the youngest person to have held the position since David Owen in 1977

29 July 2008 Writes an article for the Guardian which sparks speculation he is planning a Labour leadership bid, which he later denies

15 January 2009 Delivers an outspoken critique of US counterterrorism strategy, again in the Guardian, criticising the phrase war on terror as "misleading and mistaken"

Kate Ferguson