World Cup time means a conversation about England. The passion and pride come bursting out; so does the black humour. Whether it's the foreign coach saga, the faults of national players or the profusion of St George's flags, this is the moment we talk about England. At the general election, Labour received fewer than one in five votes in each of the three southern regions of England outside London. Almost nine in every ten voters Labour lost between 1997 and 2010 are in England. That's over four million lost English voters and 137 lost English MPs. If Labour is to avoid becoming a regional or sectional party, we need to confront the task of winning back our support among working-class and middle-income voters across England.

Labour is well placed. The Tories have no serious presence in Scotland; the Lib Dems are barely credible across swaths of the UK (think of Wales).It is only Labour - with its MPs, MSPs, AMs and thousands of councillors across Great Britain - that can speak for the whole nation. Tony Blair's connection with "Middle England" was a profound electoral attribute. It is less well remembered that early on Tony made the patriotic case for strengthening the bonds of community.

Cap doesn't fit

I know how passionately he felt. But, over time, he failed to take sufficient account and respond fast enough to the real struggles that many communities faced in confronting the impacts of globalisation - migration, low wages and public services under strain. When I campaigned in the last election it was very clear that the effect of this failure was people turning their backs on Labour. They felt we were no longer standing up for them at a time of huge change in their lives.

The easy answer is that immigration was the cause and that we should have been tougher. Yes, we should have prepared better for the initial opening of borders within the EU. We learned our lesson. But to say that if we somehow get tougher on immigration then these voters will come flocking back to Labour is wrong. We have to tackle the whole difficulty, not just part of it.

I don't want to engage in a Dutch auction on immigration. It's not my way and it is not Labour's. In fact, I don't think it's our country's way either. Just ask Michael Howard about the 2005 general election - he tried it and lost. Now we face a much bigger problem. There was not enough concern for the ties of belonging and place, which constitute so much of what makes life worth living. In many core Labour communities, a disdain for tradition and enthusiasm for hyper-modernity, constant change and all the glittering wonders of globalisation cost us votes and in some cases the BNP benefited.

Gordon Brown faced a different problem. As a Scottish prime minister confronting the Anglocentric media, he sought to emphasise the bonds of Britishness. His was a heartfelt and rigorous account of British national identity but it failed to capture the public imagination. Moreover, as Gordon was seeking to construct an idea of Britishness from above, more and more of our fellow citizens were expressing an identity bound up in the history and iconography of England, Wales and Scotland. Gordon's great achievement was to solve the Scottish question (of a Scottish prime minister governing the UK in an age of devolution), but he did not resolve Labour's English question.

Labour needs a revived politics of Englishness rooted in a radical and democratic account of nationhood. We need to draw upon a specifically English story that points to the battle for social justice born of a proud tradition of personal liberty and independence - as resentful of corporate elites as meddling bureaucracy.

The challenge is to translate a notion of Englishness into the veins of our politics. One of the defining features of Englishness is its very diversity. There are many Englands: there is the conservative England of G K Chesterton, Hilaire Belloc, and now David Cameron and Nick Clegg; there is also the leftist England by which I am inspired - of mutualism, co-operation and the epic history of places such as the north-east, home to my constituency.

Robin Hood

On policy, the English commitment to democracy, to equal life chances and to challenging corporate excess can be seen in widespread support for an elected Lords, free school meals and the "Robin Hood" tax. The politics of England are also closely bound up in questions of the governance of Britain. Labour as a party remains too centralised, too London-dominated. We must recognised that, after a strong start, our mission to devolve power and challenge the centralised British state foundered.

An "English Parliament" is not the answer. We must strengthen the civic pride and economic resilience of English towns and cities. This is how the sense of identity, belonging and place of the many Englands can be better embedded and expressed. Labour needs to work with the grain of local and institutional affiliations - from army regiments to hospitals, from fire services to local authorities.

Globalisation has brought many gains, opening up our lives to new ideas and new opportunities. But its unforgiving logic has also shattered communities and industries. The people of England demand that we not simply be subject to the logic of the global economy, but shape its possibilities for the common good.

If Labour is going to gain support outside its metropolitan heartlands and aspire to government again, it needs to speak for England and identify with its traditions and values. In four years' time, as the English football team lifts the World Cup in Brazil, Labour needs to be leading that national conversation.

David Miliband is shadow foreign secretary and the MP for South Shields.