In the 1964 general election, the Conservative candidate in Smethwick, Peter Griffiths, stood for parliament with the aid of the most offensive slogan of the modern political era: "If you want a n**ger for a neighbour, vote Labour." He won the seat. But when the new Commons assembled after the election, Harold Wilson, the incoming Labour prime minister, said Griffiths should be treated by MPs as a "political leper".

In the 2010 general election, the Labour candidate in Oldham East and Saddleworth and former immigration minister, Phil Woolas, published pamphlets with headlines such as "Lib Dem pact with the devil", "Targeted: militant extremists go for Phil Woolas", "Lib Dems in mosque planning permission stitch-up", and "Straight-talking Woolas too fair for militant Muslims". Behind the scenes, Woolas's advisers circulated emails discussing the "need . . . to explain to the white community how the Asians will take him out . . . If we don't get the white vote angry, he's gone."

Woolas won the seat with a majority of only 103. But he was not treated as a "leper" upon his return to the Commons by his parliamentary colleagues, nor was he reprimanded by the Labour leadership. Instead, the acting leader, Harriet Harman, kept him on as shadow immigration minister – a front-bench position then formalised by the new leader, Ed Miliband, in October. The shadow home secretary, Ed Balls, had no say in the matter. "It was communicated to Ed [Balls] by the leader's office that Phil would stay on," I was told.

Delayed reaction

Woolas has since been stripped of his seat in parliament by a special election court, which ruled that he knowingly made false statements about his Lib Dem opponent, and suspended from the Labour Party. Is Miliband guilty, in the words of one former minister, "of a huge mistake and misjudgement", in retaining the tainted Woolas on his front bench? "We couldn't pre-empt the outcome of the court case," says a shadow cabinet minister and close ally of Miliband. Another friend of the leader tells me: "Ed was in an impossible position – it was Harriet who had kept Phil on as a frontbencher. He had to wait for the court verdict."

Did he? The leaflets were in the public domain long before the trial. Miliband could have made an example of the odious Woolas, in the same way he stood up to the former chief whip Nick Brown. After all, he ran for the leadership on a mantra of "change" and a fresh start. "Innocent until proven guilty" is not a good enough defence – as Labour keeps reminding David Cameron over his communications director, Andy Coulson. "The Woolas debacle has rendered null and void our line on Coulson," a former Labour minister tells me.

At least Miliband and Harman acted promptly to suspend Woolas following the court judgement. Old-right Labour tribalists, however, have closed ranks around Woolas. MP Graham Stringer claimed their man has been "hung out to dry". The party's former general secretary Peter Watt said the suspension showed a "complete lack of humanity", while deigning to describe Woolas's pamphlets only as "controversial, to say the least". The veteran MP David Winnick, defending Wool­as, said merely that "he may have gone over the top" in his election campaign. Speaking at a tetchy meeting of the Parliamentary Labour Party, in which MPs lined up to denounce Harman, the backbencher Michael Connarty asked the deputy leader to "examine her conscience".

What is going on here? It is Connarty and his PLP colleagues who should examine their consciences. Woolas's inflammatory leaflets focus­ed on the need to "galvanise the white Sun-reading voters" of Oldham, in the words of one of his aides. In 2001, the town was the setting of Britain's worst race riots for more than a decade. The Ritchie Report into the ­violence accused the BNP of distributing "crude leaf­lets" in order to "stir up tensions", adding: "The mainstream political parties have a big role to play in countering this threat." Yet here was a Labour minister using BNP-style scare tactics.

Woolas has a history of making intemperate and provocative remarks on issues such as ­immigration, race and Islam. On immigration, his rhetoric earned the praise of the Sun and ­MigrationWatch's Sir Andrew Green. He once claimed his own family and children had suffered from the impact of migrant workers, without providing any details; and, in 2009, he described the Tory government of Ted Heath as "soft" for allowing in East African Asians. On race, Woolas echoed the rhetoric of the far right when he claimed, in 2003, that "racist attacks" by blacks and Asians on white people were being ignored by the authorities. On Islam, he accused veiled Muslim women of provoking "fear and resentment" among non-Muslims as well as fuelling the rise of the BNP.

Cynical populism

Woolas embodies the cynical, authoritarian populism of New Labour – the party of David Blunkett, who blamed refugee children for "swamping" schools, and Gordon Brown, who promised "British jobs for British workers". Their government locked up the children of migrants and Woolas authorised security guards employed by private contractors to use "physical control in care" techniques to deport people – including the mentally ill and children under 18 – according to documents obtained by the Liberal Conspiracy website ­under the Freedom of Information Act.

Labour can no longer afford to be seen as the "nasty party". Ed Miliband, away on paternity leave, needs to think long and hard about how to woo back disaffected liberals who left the party and refused to vote for it in 2005 and 2010. He is right to talk of "optimism", "hope" and a "new generation" – and decline the opportunity to outflank the coalition on the right. Labour cannot win a public auction to show who is toughest on minorities and foreigners, nor should it want to. This is the scorched-earth terrain of the Tories: despite Woolas's unpleasant rhetoric, Cameron's Conservatives enjoyed a double-digit lead in the polls on the immigration issue in the run-up to the general election.

The Neanderthal tendency within Labour – personified by Woolas's supporters – has been emboldened. Miliband must confront it. There can be no "business as usual" on his watch. As the backbench MP Jon Cruddas, perhaps the party's most thoughtful figure on the subject of race and immigration, has said: "If Labour becomes the voice for this sour, shrill, hopeless politics it will die. And it will deserve to."