Whisper it quietly, but the Tories won't win the next election. I say this not because of George Osborne's slashing of taxes on his rich chums - a move described by David Cameron's former speechwriter as "a basic blunder that sends a missile into six years of Tory modernisation" - nor because of the fallout from the "cash-for-Cameron" debacle.

I say it because, despite what the Tory-supporting press might have us believe, this isn't a Conservative country. Listen to the verdict of Tim Montgomerie, the plain-speaking editor of the ConservativeHome website. "Four numbers should haunt every Tory: 31 per cent, 32 per cent, 32 per cent and 36 per cent - the percentages of the vote that the party won at the last four elections," he wrote on 20 March, noting how there "may be a Conservative prime minister but the Tory brand remains weak".

It has been 20 years since the Tories last won a Commons majority and, despite losing at the last election, Labour continues to have the bigger pool of potential voters. A recent YouGov poll found that only 30 per cent of the public says it would "never" vote Labour compared to 42 per cent for the Conservatives. The Tory brand isn't just "weak", it's toxic.

Coalition has been a double-edged sword for the Conservative Party. In the early days of the Cameron-Clegg bromance, commentators marvelled at the new cover afforded to the Tories by their Lib Dem human shields.This was detoxification by proxy. Now, as we approach the midpoint of this parliament and desperate Liberal Democrats publicly obsess over "differentiation", the situation is reversed. "We keep getting painted as nasty right-wingers who are stopping what the saintly Lib Dems are trying to do," says a frustrated former adviser to the Prime Minister. Coalition government, he says, "emphasises the more traditional and unpopular aspects of the Conservatives".

On the slide

The simple truth is that Cameron needs to increase the Tory vote share at the next election if he is to secure a parliamentary majority. But not since 1974 has an incumbent prime minister pushed up his party's share of the vote. It was beyond the ability of Margaret Thatcher (in 1983 and 1987) and Tony Blair (2001 and 2005) and as a leading psephologist, John Curtice of Strathclyde University, points out: "Cameron is no Thatcher or Blair. If you look at his leader ratings, he is basically an average prime minister . . . He doesn't enthuse people."

Neither does Ed Miliband. But since he became leader, Labour has consistently led in the polls and won five consecutive by-elections. “It took the Tories several years, basically until Cameron came along [in 2005], to look electorally competitive," Curtice says. But, in 2010, "Labour got competitive again within a matter of months".

Writing off Miliband is bizarre. Few seem to realise that even after the coalition's boundary review, the Conservatives require a lead over Labour of about 7.5 percentage points in order to secure a majority. On an equal share of the vote, Labour would still win more seats - thanks to lower turnout in Labour seats, regional disparities and anti-Tory tactical voting.

Amusingly, someone who has greater confidence in Miliband's electoral prospects than most shadow cabinet ministers is Michael Ashcroft, the Tories' former deputy chairman. "Expanding the Conservative voting coalition to the point where it will elect a majority Conservative government is a strategic challenge for Mr Cameron," Ashcroft observed last May, adding: "First, he must hold together those who voted in 2010 . . . Just as important, he must attract new voters in substantial numbers."

But where will these voters come from? The three key groups that Cameron failed to charm in 2010 - public-sector workers, Scots and ethnic minorities - will, after five painful years of austerity, have even fewer reasons to vote Conservative in 2015. I'm told the PM is desperate to hire a new adviser on "BME" issues; the Tory chair, Sayeeda Warsi, can't win over black and minority-ethnic voters on her own.

Whose complacency?

According to Ashcroft, "while the Conservatives struggle to piece together two-fifths of the electorate, Labour's core support plus left-leaning former Lib Dems could theoretically give Ed Miliband close to 40 per cent of the vote without needing to get out of bed".

Montgomerie agrees that it's not Labour complacency, as is often claimed, that's the problem but "extraordinary Tory complacency". "Do not underestimate the strength of the Labour brand," he tells me.

Curtice, however, says the Tories could still have a chance of winning in 2015 if the economy turns around; he reminds me that the
Office for Budget Responsibility (OBR) is forecasting growth of 3 per cent in 2015. Perhaps. But the OBR has had to downgrade its growth forecasts four times over the past two years.

And even if the economy recovers, will it cancel out memories of the Osborne slump? Of 2.7 million unemployed? Of the biggest fall in living standards since the 1920s? Lest we forget, the Institute for Fiscal Studies says 88 per cent of spending cuts are still to come between now and 2015 - the cuts are yet to "bite".

Some gloomy Labour figures just shrug and point to the experience of the 1980s, when the Tories won a landslide in 1983 despite mass
unemployment. However, the analogy doesn't hold: the SDP-Liberal Alliance phenomenon is long gone, Argentina isn't about to reinvade the Falklands and, again, Cameron isn't Thatcher. So why does one encounter such despair and defeatism on the Labour side? The party's brand is stronger than the Conservatives'; the electoral system is biased in its favour; and Tory austerity isn't a vote-winner. There may be no room for complacency, but now is not the time for caution either.