King David struggles to control his cabinet of wayward barons

While the New Labour project was staffed with card-carrying Blairites, it is hard to identify ardent

Seventy Whitehall is not a resonant address. It doesn't evoke history and statecraft in the way that, say, 10 Downing Street does. This unassuming doorway on the road between the Treasury and Trafalgar Square, however, leads to the Cabinet Office and the Deputy Prime Minister's office, which makes it, in effect, the department for coalition - and one of the mightiest addresses in the land.

Some civil servants in Nick Clegg's office think that the building should reclaim the title of the royal dwelling that occupied the site until the late 17th century: Whitehall Palace. Then, the ambitious mandarins say, it would be recognised as a rival power base to Downing Street. They are only half-joking.

There is nothing new in competition between Whitehall departments but coalition has led to a more fundamental redistribution of power and the loser in this new model appears to be the Prime Minister. Tony Blair and Gordon Brown were absolute monarchs. This doesn't mean that their power was unchallenged. Blair came under relentless pressure to surrender the crown to his chancellor. Brown's rule was disrupted by a sequence of failed coups.

Yet, throughout the Labour years, there were only two tribes that mattered in government: supporters of the king in No 10 and allies of the Treasury pretender. Other realms were carved up in reshuffles and reorganisations, bestowed on docile ministerial princelings.

Raiding parties

David Cameron promised an end to the capricious redrawing of Whitehall boundaries, which he rightly saw as needlessly disruptive. As for reshuffles, coalition limits the available permutations, since cabinet jobs must be parcelled out between two parties. This is causing indignation among ambitious Tory MPs, who fought the election expecting swift elevation up the ministerial ladder and now see it teeming with Lib Dems. The wilier ministers have adapted to the new, more rigidly demarcated political landscape, digging themselves in, paying the necessary homage to the king in No 10 but not allowing themselves to be controlled by him. The age of absolute monarchy has given way to the age of the cabinet barons.

George Osborne is hugely powerful but the Treasury has always been a kingdom in its own right. The strongest new player is Clegg, the yellow baron, whose department exists only to limit Cameron's room for manoeuvre. Even the Downing Street policy unit has been infiltrated so that, in the words of one senior official, "There's the blue team on one side and then, on the other side, there's the yellow team man-marking them."

It would be complicated enough having two party leaders in the cabinet but, if you count the former Tory bosses William Hague and Iain Duncan Smith, there are four. Hague is a peaceable baron, content in his Foreign and Commonwealth Office realm. Duncan Smith, meanwhile, driven by an evangelical ambition to reshape society through changes to the welfare system, is more prone to launching border raids on neighbouring castles. "IDS has managed to have fights with every single member of the government," is how one adviser puts
it. "It's quite impressive."

After the summer's riots, for example, Duncan Smith moved to annexe policy for "problem families" from Michael Gove at the Department for Education. Gove, who is more subtle than IDS in his exercise of baronial independence, but no less effective, resisted. After several weeks of skirmishing, a bizarre deal was brokered by No 10, in which responsibility for domestic dysfunction was farmed out to Eric Pickles, a neutral baron.

There are minor Lib Dem barons: Chris Huhne, Energy and Climate Change Secretary, and the Business Secretary, Vince Cable. Neither is a natural party ally of Clegg, which sets them at two removes from Cameron's writ. Then there is the Justice Secretary, Ken Clarke, who is an informal ally of the Lib Dems, because of his pro-Europe instincts and liberal penal policies. At 71, Clarke is so far beyond ambition as to be immune to discipline.

The list could go on to include at least half of the cabinet. This is not rebelliousness in the conventional sense. It is self-sufficiency - a kind of indifference to the Prime Minister. "It's quite hard for the PM to push these people to do what he wants them to do," one senior official says. Or, as an adviser to a usually loyal minister declared to me, recently: "We don't really pay too much attention to what No 10 thinks."

Idle hands

This shortening of prime ministerial reach has a number of causes. Coalition is a big one; Cameron's lack of interest in policy detail and short attention span are others. ("You sometimes get the feeling that what he says on Monday depends on the person who wrote his speech on Sunday," one insider says.) Another factor is Cameron's failure in opposition to recruit true believers beyond a tiny clique of friends. While the New Labour project was staffed with card-carrying Blairites, it is hard to identify ardent Cameroons in government.

This is going to be a problem for the Prime Minister, not least at his party conference in Manchester. The Tories are increasingly frustrated by what they see as the failure to deliver an authentically Conservative agenda in government beyond fiscal austerity. They see Lib Dems boasting of policy vetoes; they observe high-profile U-turns; they recall old puzzlement about Cameron's personal beliefs and, like everyone, they worry about the economy.

The Conservative party conference will be well stage-managed as usual but, in the background, there is a rising sense of panic that the Cameron project is also somehow stagnating. This is peculiar, given that frenzied reform projects are under way in pretty much every major department - welfare, health, schools, justice. Those, however, are the realms of the barons. The king, when not attending to foreign adventures, looks strangely idle.

That is not sustainable. Before too long, the monarch will eye his surrendered powers jealously and start clawing them back. It will then be worth recalling what was on the site of 70 Whitehall when it was first incorporated into a Royal Palace. In Tudor times, the place where the Lib Dem leader now has his office was a venue for cockfights.

Rafael Behr is chief political commentator of the New Statesman