NS Essay - 'The Tories will briefly return to power in 2010, but after that it will be Labour's century'

The Blair-Brown style, with its notorious control-freakery, is the result of too long in opposition.

It is not the best of times for Labour. With former cabinet colleagues exploding like cluster bombs, crucial votes being won with support from disloyal Conservatives, party membership falling and Labour polling only slightly ahead of Michael Howard's motley crew, the project looks endangered.

But the long-term prospects for the party are in fact better than at any point in its history. The 21st century could herald a reversal of the pattern of the 20th, in which Labour's principal role was to give the Conservatives the odd break from the rigours of government. Even if, as seems likely, the Tories get back into office fairly soon, Labour will remain the party of power. This should be the Labour century.

But such dominance is far from guaranteed. One of the oddities of the current political climate is that the UK has no natural party of government. The Conservatives lost the crown in the quagmire of the Major administration. But Labour has yet to inherit it. Tony Blair and Gordon Brown have performed extraordinarily well in government, especially at the helm of the least experienced cabinet in British history. But they remain insecure incumbents.

Such achievements as macroeconomic stability, a minimum wage, significant public sector investment, devolution, Bank of England independence, the New Deal and Sure Start are easy to understate or forget. These are real, and mostly irreversible, achievements. The Blair/Brown problem has not been lack of competence but lack of confidence. The key to understanding their psychology is not the absence of red boxes from their lives before 1997 - it is the long, lonely, enervating years of opposition and the trauma of 1992. Neal Lawson, head of Compass, a new Labour pressure group, says that the 1992 defeat "is hard-wired into their political make-up; it is impossible to understand Labour without understanding that".

Fourteen years of opposition cast a long shadow over Downing Street. Much of the Labour administration has been marked by a paranoid fear of its fragility and imminent end. Labour leaders could not believe how many seats they won in 1997 - and they never really did come to believe it. Although "we are the servants now" was a soundbite, and although Blair and Brown are often criticised for control-freakery and centralism, there has always been a sense that members of Labour's high command have never come to feel like the masters.

Blair and Brown have laid the foundations for a century of Labour dominance by abandoning socialist dogmas, winning elections and governing competently and without corruption. But they are incapable themselves of turning Labour into the natural party of government. For them, government is a long-sought, hard-won victory that is always at risk. The leaders of a natural party of government assume they have the freehold on Downing Street; Blair and Brown feel and act like leaseholders.

This may sound like faint praise. But it is a fact of life that leaders who rescue failing organisations are different, in their histories and personalities, from those who lead them to greatness. This is a lesson that Blair's business friends could pass on: "rescue" CEOs have to be succeeded by "investment" CEOs, who are not scarred by the battles of the past. Labour cannot become the natural party of government until its leadership passes to a generation unscarred by opposition.

This seems unlikely to happen in the near future. Labour will comfortably win the next election, albeit with a significantly reduced majority. Gordon Brown will get his moment in the sun, but it will be a short one: the Conservatives, led by a younger, wetter leader, will win in 2010. Labour will then be back in opposition - and the true test will begin. A Conservative government is likely to be fairly hopeless, given that the cabinet will comprise a mixture of has-beens and overrated ingenus. For all their recent presentational successes, the Tories have no strength in depth.

By contrast, the Labour front bench of 2010-2015 will pack some serious punch. Labour has a cadre of thirtysomething MPs who will be able to use the rest-cure of opposition to develop a robust political philosophy that can guide them, in power, for a generation. And so, I predict, the years 2015-2030 will be Labour ones again.

Long-term success depends on this generation - a point that Blair should bear in mind in his next reshuffle. Look at the box (right) forecasting the Labour cabinet of 2015 and first at the sextet of David Miliband, Yvette Cooper, Douglas Alexander, Ruth Kelly, David Lammy and Stephen Twigg. All of them are already ministers (half of them just one rung below the cabinet) at an average age of 35, and, although they lack certain attributes, confidence is not one of them. Look next at Ed Balls, Ed Miliband and Geoff Mulgan waiting in the wings, and if you are a Conservative, you should be very afraid. This generation spent its teens under Margaret Thatcher and came professionally of age as Labour took power. These people have experienced opposition only indirectly, but government at first hand. As a result, they are not scarred by, or scared of, opposition in the same way as their bosses. They can make this the Labour century.

To succeed, they need to meet four challenges. First, they have to survive and profit from the inevitable period of opposition. In the past, Labour has nearly always imploded in opposition and swung sharply left in reaction to the perceived sell-outs of the previous administration. Factionalism, finger-pointing and poison are Labour's opposition fare. "The party has an oppositional mindset," says Sunder Katwala, general secretary of the Fabian Society (and another possible future star), "so the challenge will be to remain serious about power even when it is out of government. There will be those who argue that Labour has been thrown out because it has not been left-wing enough and will want to take on the technocrats." The Milibands, Alexanders and Coopers will have to win the battle for control of the party in opposition, and avoid the cul-de-sac of betrayalism. At the moment, few left-wingers of their generation look to be of sufficient calibre to make this a difficult task - but history cautions against complacency.

The second challenge is to avoid the filial bitterness - the paranoia, secrecy, suspicion and grudge-holding among senior ministers - that has disfigured the past seven years. It is not just Blair and Brown. John Prescott refused for years to meet Harriet Harman, who was also reviled by Alastair Campbell and his partner, Fiona Millar, who were distrusted by Peter Mandelson, who struck fear into everyone, and so on.

These dysfunctional quasi-familial relationships were largely the product of the long, frustrating years in the wilderness. The next generation can roughly be divided into Brownites (Douglas Alexander and Ed Miliband, for example) and Blairites (Geoff Mulgan and David Miliband, for example), but the big question is whether they avoid their elders' viciousness or re-enact it.

The third challenge likely to re-emerge for Labour in opposition (and perhaps even in the smaller-majority term of 2005-2010) will be the familiar one of whether to work more closely with the Liberal Democrats. Blair was keen to play up the prospects of reuniting the liberal and socialist strands of British political history in a unified anti-Conservative movement. Such a realignment, he suggested, was necessary to ensure a "progressive century". He even blew hot about proportional representation. But that was in 1995. One Labour landslide later and Lib-Labbery was quickly defenestrated.

The death of the Lib-Lab dalliance brought distress to many thoughtful progressives in both parties - but there were plenty more who were relieved. They recalled the 1903 pact, in which the Liberals agreed that some Labour candidates could run unopposed in the 1906 election, and so allowed Labour to become a serious electoral movement. Jeremy Thorpe's view, that this allowed the "socialist cuckoo in the nest", was echoed by many senior Labour figures who opposed offering similar assistance to the Lib Dems.

The temptation to reopen the flirtation will be especially strong, given the liberal instincts of many of Labour's rising stars, the Lib Dems' shift towards social democratic positions and the renewed fears of Conservative hegemony. But the temptation should be resisted. The drive towards any kind of alliance has always been defensive, which is why it reappears only when Labour is in opposition or has a small majority. It is based on a shared desire to beat the Conservatives - honourable, sure enough, but a lousy basis for an alliance. A realignment in opposition will reek of opportunism and desperation, to the electorate.

Yet Blair, Roy Jenkins and others are right to say that the breach between liberalism and socialism needs to be healed. The solution, however, is not a marriage of parties but a reinvention of Labour from within. To become the natural party of government, Labour needs liberalism but not the Liberals. For a start, there are growing numbers in the Labour ranks who have adopted a liberal mantle, and who in some cases possess a genuinely liberal instinct. Peter Hain, David Miliband, Tony Wright and others describe themselves as "liberal socialists". Meanwhile, the Liberal Democrats' claim to the inheritance of pure liberalism has become pretty shaky: they advocate such policies as making people take out compulsory second pensions to supplement their basic state pensions.

Neal Lawson, one of the early advocates of cross-party collaboration, now says: "I was convinced that Labour needed a shock to the system to move in a more progressive, open direction and that the shock had to come from outside the party. It may be that Labour can do it without that."

So this is Labour's fourth and biggest long-term challenge - to bring social democracy to a liberal nation. And it does not require going cap in hand to other parties. The motto needs to be "liberal socialism in one party".

There are signs that Labour's younger generation is thinking in more liberal directions. Alexander, in a forthcoming Fabian Review article, describes Labour's values as "solidarity, equality and so liberty". In a speech given before he became a minister, Miliband talked of a "social democratic commitment to social justice through collective action enriched by commitment to individual freedom in the market economy". There is talk of "empowerment" as the party's next big theme. And freedom of choice is a leitmotif of public service reform.

These are just straws in the wind. The hard work of crafting a modern liberal socialist agenda - in time for the centenary of L T Hobhouse popularising the term in 1911 - has hardly started. Instead, the debates are around ill-defined issues such as "new localism", which, as Katwala points out, are "hyper-wonky" and a long way from a broader narrative about Labour's goals.

For liberal socialism to get off the ground, Labour needs to recognise clearly that individual freedom and social objectives can conflict. Rather than assuming that the conflict can be dissolved by vacuous "win-win" rhetoric, it needs to focus on how they can be sensibly managed. Labour has to wean itself from its instinctive statism (even Katwala of the Fabian Society says that "Labour probably needs to be a bit less Fabian"), place greater trust in individuals and live with some of the resulting messiness.

Above all, liberal socialism needs to focus as much on means as ends. Labour politicians place much greater emphasis on what drops out at the end of the production line than on the productive process itself - "what matters is what works". But as Hobhouse put it: "If there be such a thing as a Liberal Socialism - and whether there be is still a subject for inquiry - it must be . . . democratic. It must come from below, not above . . . It must engage the efforts and respond to the genuine desires not of a handful of superior beings but of great masses of men."

One of the paradoxes of new Labour has been the practical centralisation of power alongside its structural devolution. There are now signs of more authentic commitment to the devolution of power, especially down to neighbourhood level. And although it is easy to snigger at the "big conversation", it is a sight better than nothing. But there is a very long way to go.

In 1966, Richard Crossman complained that "the notion of creating the extra burden of a live and articulate public opinion able to criticise actively and make its own choices is something which most socialist politicians keenly resent". The cabinet in which Crossman served held attitudes very different from those of the current cabinet - look at the devolution that has taken place - but his remark still captures a kernel of quiet anti-democracy within the Labour movement.

As David Marquand has said, social democrats instinctively view society as a patient upon which they, the surgeons, operate. This is perhaps the only outdated Labour legacy that Blair has failed to ditch. But its abandonment is crucial to any chance of a philosophy that gives Labour a surer footing. Liberals usually worry about what government should not do, socialists about what it should. Liberal socialism will stand or fall by its attention to the way government acts.

The hope is that Labour's next generation will be confident enough of its grip on power to loosen it in the service of democracy, robust enough to weather a louder public conversation, and brave enough to let flowers bloom. If it lives within the limits set by this generation - limits set so that Labour could become the party in government but woefully inadequate for a party of government - the Tories may re-emerge as the freeholders of Downing Street. But if the new generation can see 1997-2010 as just the opening chapter, then the keys to Downing Street, and the century, could belong to Labour.

Labour cabinet 2015

Current position in parenthesis

Prime Minister: David Miliband (School standards minister)

Deputy PM: Yvette Cooper (Minister in Deputy PM's Office)

Chancellor: Ed Balls (Adviser to Gordon Brown)

Leader of the House: Phil Woolas (Deputy Leader, House of Commons)

Foreign Secretary: Peter Hain (Leader, House of Commons)

Home Office: Geoff Mulgan (Head of No 10 Policy Unit)

Environment: Jacqui Smith (Industry and regions minister)

Int Development: Jon Cruddas (MP for Dagenham)

Work and Pensions: Ed Davey (Lib Dem front-bench spokesman)

Transport: Ed Miliband (Chair, Council of Economic Advisers)

Health: Patricia Hewitt (Secretary for Trade and Industry)

Defence: Martha Lane Fox (Co-founder of lastminute.com)

Chief Secretary: Stephen Twigg (Schools minister)

Leader, House of Lords: Gordon Brown (Chancellor of the Exchequer)

Trade and Industry: Ruth Kelly (Financial Secretary to the Treasury)

Education: Hilary Benn (International Development Secretary)

Culture, Media, Sport: David Lammy (Minister for constitutional affairs)

Chief Whip: Ben Bradshaw (Nature conservation/fisheries minister)

Party Chair: Oona King (PPS to Patricia Hewitt)

Lord Chancellor: Douglas Alexander (Minister for the Cabinet Office)