The Labour Party, founded in 1900, has been in existence for 112 years. But there have been Labour governments for just 32 of those years; for another eight years, Labour participated in wartime national unity governments. During the remaining 72 years, it was in opposition. Why has so much of Labour’s existence been spent in opposition? There are three reasons.

The first is factionalism. After the election defeats of 1951 and 1979, Labour was rent by fratricidal disputes between left and right – between the Bevanites and the leadership after 1951 and between the Bennites and the leadership after 1979. The party seemed more anxious to have a dialogue with itself than a dialogue with the British people, who decided, both in the 1950s and in the 1980s, to confirm Labour’s status as a party of opposition.

After the defeat of 1970, too, Labour threatened to descend into factionalism, but Harold Wilson skilfully contained it. Even so, the party returned to power in March 1974 through the vagaries of the electoral system, as opposed to any merits of its own. Between 1970 and the general election of February 1974, Labour lost 6 per cent of the vote – one-seventh of its support – the largest fall in votes of any opposition party in Britain until, in 1983, Michael Foot succeeded in doing even worse.

Labour’s defeat in 2010 was worse than those of 1951, 1970 or 1979. Indeed, in terms of percentage of the vote, Labour did worse in 2010 than at any time since it became a mass party except for 1983. Yet there have been no renewed outbreaks of factionalism, no divisions between left and right. The party has shown a remarkable degree of unity. There have been no latter-day equivalents of the Bevanites or the Bennites. That, no doubt, owes much to Ed Miliband’s empathetic and consensual style. It is an unnoticed achievement.

The second reason for Labour’s long years in opposition is that it has, since 1918, seen itself as the sole party of the left, and has been intolerant of competition from other claim ants. In the 1920s, it was more eager to eliminate the Liberals than to seek a progressive alliance with them, an alliance that might well have undermined Conservative dominance. Some Labour leaders actually preferred the Conservatives to the Liberals. Ramsay MacDonald, Labour’s first prime minister, declared in 1924 that he “could get on with the Tories. They differed at times openly then forgot all about it and shook hands. They were gentlemen but the Liberals were cads.”

In the 1930s, the Labour leadership stood firmly against a Popular Front, an alliance with Liberals and anti-appeasement Conservatives which might have brought about a change of government policy. Indeed, Labour succeeded in its aim of driving out the Liberals as a third party, but the price was a long period of Conservative hegemony, and the 20th century became the Conservative century even though there may well have been a progressive majority among the voters for much of the period.

Liberal friends

In the 1980s the third force revived – first the Liberal-SDP Alliance, then the Liberal Democrats. Labour’s instincts remained the same: to regard the third force as a competitor rather than a potential ally. Tony Blair was a great exception. The Ashdown diaries show that he would have preferred a coalition with the Liberal Democrats to a single-party Labour majority. Had the majority in 1997 been smaller, Blair would almost certainly have sought coalition.

Gordon Brown also sought an alliance with the Liberal Democrats in 2007 after becoming prime minister, and suggested coalition to Menzies Campbell, the then Liberal Democrat leader. In 2010, after the election, he again offered coalition to the Liberal Democrats, saying that the moment had come to create the progressive alliance. But it was too late. Labour was too weak. In 1997, it had been too strong. It was never the right moment.

Here, too, Miliband can claim to have escaped the entrenched positions of the past. There are no precise details of what has transpired between him and the Business Secretary, Vince Cable, but it is clear that a dialogue has begun and that Miliband does not regard all Liberal Democrats as the enemy. There is, after all, a powerful social-democrat tradition among the Liberal Democrats, one wing of which, including Cable, was in the SDP, a breakaway from Labour in 1981. Cable, indeed, began his political career as a Labour activist, and would probably have continued in the Labour Party had it not swung so far to the left after the election defeat of 1979.

Clearly, many Liberal Democrats were prepared to swallow their doubts about Conservative methods of curing the deficit and proposals for student fees in order to secure cherished measures of constitutional reform – reform of the voting system and a directly elected House of Lords. Now that it has become clear that these reforms are unattainable, they may well revert to their natural home on the left.

Miliband needs perhaps to make a sharper distinction between the Liberal Democrat leadership, or the Clegg-Laws faction of the leadership, whose ideological sympathies lie with the right, and the vast majority of Liberal Democrat members and voters, whose heart remains on the left. Franklin Roosevelt, after all, never attacked the Republicans, only the Republican leadership.

Yet Miliband’s flexibility towards the Liberal Democrats is a second achievement of Labour in opposition. He needs to become the leader of all progressive forces in Britain, not just the Labour Party.

There is a third reason why Labour has spent so much of its life in opposition. It is that its inclination in defeat is to retreat to its comfort zone, its core basis of support, such as that represented by the Blue Labour tendency, set up in 2009 by the academic Maurice Glasman, who was subsequently made a peer. The instincts of the Blue Labour faction are the same as those that led Labour to elect Foot as leader in 1980, and that lay behind its failure to pressure Attlee to retire after the 1951 defeat.

During the 1950s, Labour’s vote fell steadily from the 48.8 per cent peak of 1951 – achieved, paradoxically, in an election in which the party was defeated by the Conservatives. Not until after the third election defeat in 1959, however, did Hugh Gaitskell feel able to attempt to modernise the party by proposing the deletion of Clause Four from Labour’s 1918 constitution, which committed it to the nationalisation of the means of production, distribution and exchange. Even then, the party refused to support him, and it was left to Gaitskell’s successor, Harold Wilson, to modernise Labour by more surreptitious means, Wilson’s 1964-70 government being revisionist in practice if not in theory.

After the 1983 defeat, Neil Kinnock took over where Gaitskell had left off, but he was constrained by activists and his progress in modernising the party was painfully slow. It was not until 1992, after the fourth successive defeat, that Blair was able to take Labour by the scruff of the neck. Succeeding to the party leadership in 1994, he got rid of Clause Four and founded New Labour. There was just one Labour tradition that he hated, Blair told the 2006 party conference in his farewell speech – losing elections. Some in the party have never forgiven him for breaking that tradition.

Today also, there are siren voices saying that Labour should return to its comfort zone. For the essence of the Blue Labour faction is that the party should become more conservative, more respectful towards the supposed values of working-class communities and of working-class attitudes (some would say prejudices) on immigration and crime. The politics of nostalgia would be disastrous for the left, however, ideologically and electorally.

Blue Labour emphasises co-operatives, mutuals, friendly societies and more localised provision of public services. Never mind that it was because this very approach was insufficient that past Labour governments strengthened central government. It is fashionable to decry the state, but it is to the state, not the Salvation Army or the Co-op, that we turn when we find ourselves sick or out of work.

The truth is, as the recent TUC conference has shown, that there are no hermetically sealed working-class values. The values of the organised working class reflect wider social ethics. The calls at the conference for a general strike were an attempt to use the market power of organised labour to alter the policy of the democratically elected government. Miliband was right to distance Labour from it. But was the call for a strike any worse than the threat made by bankers and their like that any attack on their bonuses would lead to them taking their business abroad? If the rich and powerful can use their market power to threaten the government, why should organised labour not follow their lead?

Labour was founded more than a century ago to represent the organised working class, and perhaps its electoral rise up to 1951 and its decline thereafter can be correlated with the rise and fall of that class. Even so, the party’s central aim was to secure certain values, to transcend a society based on economic self-interest. Lab - our, Keir Hardie insisted, attacked a system not a class. If the values of community and fellowship had already been present in working-class communities, as some of the advocates of Blue Labour seem to imply, there would have been no need for a Labour Party. It was precisely because these values were not present that the party was founded.

The better life

There is a great danger in romantic mythologizing of working-class communities, a tendency sometimes practised by those who have never lived in them. Margaret Thatcher, who in the 1980s seemed to understand working-class aspirations better than Labour, declared that she had rarely met anyone from such communities who did not wish to escape from them.

Most working-class people want their children to have a better life than they had – a better education, better housing and a more fulfilling job – not to replicate their position in the “community”. Those aspirations received little expression at the TUC conference. They need to be expressed by the Labour Party. Indeed, it is only when Labour has been able to express such aspirations, as in 1945, 1966 or 1997, that it has been successful electorally.

Blue Labour, by contrast, is conservative, but conservative about a society that no longer exists, or perhaps never existed. It resembles Tory paternalism of the Baldwin-Macmillan variety; or perhaps the feudal socialism that Marx ridiculed in his Communist Manifesto as “half lamentation, half lampoon; half an echo of the past, half menace of the future”.

Speak for the middle

In any case, the old-style working class is not Labour’s problem. Even in 2010, the Conservatives were unable to win a single seat in any of the large cities of the Midlands or the north – Birmingham, Bradford, Edinburgh, Glasgow, Leeds, Leicester, Liverpool, Manchester, Newcastle, Nottingham and Sheffield, while the Liberal Democrats won just six. In fact, one reason why the remnants of the working class have so little political leverage is that they are largely concentrated in safe Labour seats. The best way to strengthen the electoral influence of the working class would be to reform the electoral system so as to eliminate the safe seat.

But Labour’s electoral problem lies elsewhere. It lies, as in the 1980s, in its failure to retain the allegiance of aspirational voters in the south of England. South of the Severn-Wash line outside London, the party holds just ten out of 197 seats. It has no MPs at all in Cornwall, Somerset, Wiltshire, Dorset, Sussex, Kent, Essex, Norfolk, Cambridgeshire, Northamptonshire, Rutland, Warwickshire, Buckinghamshire, Hertfordshire, Gloucestershire, Worcestershire or Here fordshire. The BBC’s electoral analyst David Cowling was right, therefore, to describe the outcome in 2010 as “the dismem - bering of New Labour’s 1997 electoral triumph”. Labour must seek to reassemble that triumphant coalition, not retreat to the safety of its heartland.

Labour’s problem is not the working class but the southern working class, the most aspirational segment of the working class. Indeed, psephological studies have shown that a northern middle-class voter is more likely to vote Labour than a southern working-class voter. Blue Labour would do little to win back these lost voters.

Ed Miliband has kept Labour away from its comfort zone and his call to defend the “squeezed middle” resonates with many voters. However, he has not yet allied the party firmly enough with the aspirations of voters in the south of England; and he has still to spell out his programme for responsible capitalism, taming the markets, rebalancing the economy and achieving greater fairness at a time when public spending will be constrained. Perhaps his greatest difficulty is that most voters find it difficult to identify with him, to “place” him; and, in so far as they do place him, it is as a north London intellectual, remote from their concerns. His background is not his fault any more than David Cameron’s is, but he needs to transcend it.

Wilson faced a similar problem. Before becoming Labour leader, he had been associated in the public mind with the bureaucratic restrictions necessary after the war and a dry, impersonal, economic approach to politics. To overcome his image problem, he enlisted advisers such as the journalist Joe Haines and Albert Murray, the MP for Gravesend, who could supply what he lacked. Wilson rapidly reinvented himself as a man of the people.

Miliband needs to do the same. He could begin by avoiding terms such as “predator capitalism” and “predistribution”, which may resonate with readers of the New Statesman, but lend themselves to ridicule elsewhere. He needs to become the natural spokesman of Middle England, the “squeezed middle” whose aspirations he has sought to champion.

Democracy, it is often said, is government by explanation, and the crucial electoral battleground is that of public opinion. Governments can transform opinion by acting. Oppositions can transform opinion only by speeches, by teaching. For many years, the left in Britain has lacked a teacher. The task for Ed Miliband is to show that he can be as formidable a teacher for the left as Thatcher was for the right.

Vernon Bogdanor is a research professor at the Institute for Contemporary British History, King’s College London. His books include “The Coalition and the Constitution” (Hart, £20), published last year.