Photograph: Getty Images
Show Hide image

The shadow power list

Who really runs Britain? The new establishment is unelected, often unaccountable and in charge of ever more of our public services.

Rafael Behr writes: One of the least persuasive pledges made at the last general election was the Conservative offer to “shrink the state”. In a focus group conducted for the party to help it understand why it had failed to win a majority in parliament, one flummoxed voter trying to decipher what the Tories had in mind offered: “Lopping off Cornwall . . . ?”

Most people do not contemplate the state. Of those who do, few dwell on its proportions relative to some abstract, miniature ideal. In real life, power in Britain is not contained within boundaries easily definable as “government”. With most of the economy privatised, the spectre of extreme political control – the dawn raid by jackbooted government agents – is confined to science fiction and the nightmares of paranoid libertarians. We are not a tyrannised nation. Where we experience the humiliation of powerlessness this is as likely to be at the hands of a private company as a state institution. When it is a state service, there is every chance its functions have been outsourced to a private provider. If Kafka’s Josef K were looking for justice in a labyrinth of 21st-century British administration he would find its walls marked with Serco and Capita logos; his guards would be wearing G4S uniforms.

It is sometimes supposed that the opposite of centralised power must be devolution, but Britain has found a different way. The control that once belonged to government departments, historic institutions or household names has been farmed out sideways. It resides on the boards of companies no one has heard of, in quangos, in hedge funds, in networks of friends and former ministerial advisers who work for charitable bodies with opaque remits.

It no longer makes sense to speak of “the establishment” as it did in the days when the lord chamberlain could strike obscenity off the stage. The idea of the establishment survives more in the aspiration to show defiance than the craving to belong. Nowadays even Conservative ideologues drape themselves in supposed anti-establishment kudos. They imagine their public-service reforms as subversive assaults on crusty old monopolies: the quasi-privatisation of the schools network; the spread of market forces through the NHS; the drip-drip of ministerial hostility to a BBC funded by the licence fee.

One consequence of having an outsourced establishment is the lucrative opportunities it creates for lobbyists. When the government’s role is reduced to commissioning public goods, go-between agents can scoop up power and influence to match public-sector/ politician buyer and private-sector seller.

Another long-term trend is the rise of marketing and communications experts into the top tier with establishment status. It is the natural product of a liberalising ideology that sees consumer choice as the model mechanism for effective delivery of public goods. Candidates are products and parties live or die according to the health of their brand. It is typical of the age that our Prime Minister, educated at Eton, a scion of the aristocracy, had a career in public relations before politics.

Downing Street will always be at the centre of the action but no longer at the apex of a tidy pyramid of departments and offices arranged in evenly cascading hierarchies of power and prestige. What now passes for the establishment is amorphous and anonymous; a baggy blur of the commercial, the political and the ill-defined space in between. Below, the New Statesman considers just a few of the people who hold the very British brand of inconspicuous power.

Christopher Hyman

Chief executive, Serco

The National Nuclear Laboratory, the Docklands Light Railway, immigration detention centres, the London cycle hire scheme, NHS Suffolk, the National Border Targeting Centre, air-traffic control services, waste collection for local authorities, maintenance services for ballistic missiles, government websites, prisons and a young offender institution – there is almost no branch of government that has not been penetrated by Serco, the outsourcing behemoth. And few have benefited more from the growth of this shadow state than the company’s chief executive, Christopher Hyman.

In 2010, Serco, which gets over 90 per cent of its business from the public sector, paid him a salary of over £3.1m. According to research by One Society, this was “six times more than the highest-paid UK public servant [and] 11 times more than the highestpaid UK local authority CEO”.

Hyman joined Serco in 1994 following stints at Arthur Andersen and Ernst & Young and was made group chief executive in 2002. Born to an Indian family in apartheid South Africa in 1963, the abstemious Hyman considered a career as an athlete after running 100 metres in 10.8 seconds but stopped after concluding that he would never win gold. He now races Formula 3 motor cars and cried after finishing fourth in his first-ever competition. “I felt such a failure – I was embarrassed and incredibly emotional.”

A devout Pentecostalist – he fasts every Tuesday and donates a biblical tithe of his income to his local church – Hyman was at a meeting of Serco shareholders at the World Trade Center when the first plane struck on 11 September 2001. He later said of the event: “It confirmed my faith. It renewed my zest for getting the balance right and made me realise that time is not always your own.” In addition to running Serco, Hyman has found the time to release an album of gospel music, an achievement possibly attributable to his decision to sleep just four hours a night.

But while he is celebrated for being the human face of outsourcing, his company’s reputation has become increasingly toxic. In September 2012, Serco was forced to apologise after admitting it had presented false data on 252 call-outs to its out-of-hours NHS general practitioner service in Cornwall. On one occasion, a single doctor was on call for roughly 500,000 people across the county.

Having recently won the £140m contract to run NHS community services in Suffolk, Serco is likely to come under further scrutiny. As the chief executive of G4S, Nick Buckles, learned to his cost, the rulers of the shadow state can quickly become hate figures when their promises of “efficiency” prove illusory. With Labour determined to hold those in the business of NHS reform to account, don’t be surprised if Hyman finds himself hauled before a parliamentary select committee before the end of the year.

Sam Laidlaw

Chief executive, Centrica

Sam Laidlaw, of the privatised utility company Centrica (formerly British Gas), has been described as the “aristocrat” of the energy industry – and his family history indicates how the British ruling class has adapted over the course of a century, from empire to social democracy and the free market. His grandfather Hugh was an executive of the Anglo-Persian Oil Company in India, a forerunner of BP; his father, Christophor, worked his way up through BP to become deputy chairman; Sam attended Eton, Cambridge and the elite business school INSÉAD in Fontainebleau before launching his own oil career. In 2006, he was recruited to Centrica from the US company Chevron.

Laidlaw, who lives in Chelsea, has said he would like to be remembered as “someone who was good at creating businesses . . . someone people enjoyed working with, who was fun and made some small contribution to society”. He has presided over Centrica at a time when profits for energy companies have been rising steeply – along with customers’ bills. In 2011, a bonus of £848,000 raised his pay to £4.3m. In 2008 he was one of several executives denounced as “fat cats of British industry” at a Commons business select committee hearing. But as he told staff in an email, “I am not about to apologise for making a healthy profit.”

Until the end of 2012, he was a member of David Cameron’s Business Advisory Group, a collection of leaders from “sectors of strategic importance to the UK”, which gathered to provide “regular, high-level advice on critical business and economic issues facing the country”. His departure in December came, a report in the Guardian suggested, after public anger had forced the government to criticise “opaque” pricing and tariffs by energy companies, including Centrica.

Laidlaw remains an influential figure, however. He is a non-executive director of HSBC Holdings and sits on the bank’s group remuneration committee, whose responsibility it is to approve company policy on pay for senior executives – including bonuses.

Professor Malcolm Grant

Chairman, National Health Service Commissioning Board

The overhaul of the NHS by the coalition government produced another super-quango (this after the government promised to banish them). Tasked with chairing the newly minted NHS Commissioning Board is Malcolm Grant, whose official job is to “provide strategic leadership and ensure proper governance” but who will also be steering the most controversial transformation in the health service since its creation. Grant played a critical role in recruiting the non-executive and executive members of the board who between them will be managing an annual budget of £95bn. Roughly £65bn of this will be spent by clinical commissioning groups, which are replacing the old primary care trusts – but in essence, from April this year, Grant will have oversight of the entire NHS budget.

Inevitably, it’s not his only job. Grant, who grew up in Oamaru, New Zealand, trained as a barrister, has been a professor of land economy and a professor of law, and is now provost of University College London. He has had his share of chairmanships, too, running the Local Government Commission for England (1996-2001), the Agriculture and Environment Biotechnology Commission (2000-2005) and the Russell Group of research universities (2006-2009).

As if that weren’t enough, he is also, by appointment of the Prime Minister, a business ambassador for Britain and a member of the Higher Education Funding Council for England. As such, he has influence in multiple public spheres and has now been entrusted with perhaps the greatest responsibility of all: the nation’s health.

Tim Allan

Chief executive, Portland PR

Offered the chance to become the prime minister’s director of communications, few in the world of public relations would gratefully decline. Yet “no” was the answer Tim Allan gave to Tony Blair after Labour’s third successive general election victory in 2005. Correctly calculating that Blair would be unable to fulfil his pledge to serve a full term and that the role would be short-lived, Allan chose instead to remain as managing director of Portland, the political consultancy and public relations agency he had founded in 2001. Eight years after making the decision, he is unlikely to regret it.

Portland, whose clients have included Tes - co, Google, the Russian government, Coca- Cola, BTA Bank of Kazakhstan, McDonald’s and Barclays, has made Allan one of the most influential PR men in the country and one of the wealthiest. In 2012, he sold his majority stake in the firm to the US marketing giant Omnicom in a deal estimated at £20m.

Allan’s break came in 1992 when he was headhunted by Blair, the then shadow home secretary, after studying at Cambridge. On the recommendation of one of his researchers, James Purnell, a friend of Allan’s from the Royal Grammar School in Guildford, Surrey, Blair invited the young graduate to join his office. As Allan later recalled: “ ‘Were you involved in student politics?’ Blair asked me. ‘I’m afraid not,’ I said. ‘Great. Can you start tomorrow?’ he responded.”

He was promoted to deputy press secretary in 1994, working directly under Alastair Campbell, and became deputy director of communications after Labour’s 1997 election victory. He left Downing Street the following year to become BSkyB’s director of corporate communications, a role that involved writing speeches for Elisabeth Murdoch, and founded Portland after winning the BSkyB PR contract from Bell Pottinger.

With his Conservative connections, Allan has adapted to life under the coalition better than most Blairites. His first employee at Portland was Rachel Whetstone, whom he hired from Carlton where she was working alongside David Cameron, the TV company’s communications chief. Whetstone, who is now head of communications for Google, later married Steve Hilton, Cameron’s director of strategy between 2005 and 2012.

Allan’s other Conservative hires have included the Prime Minister’s former press secretary George Eustice and his former director of policy James O’Shaughnessy, currently chief policy adviser at Portland. And among recent recruits are Allan’s former No 10 colleague Campbell as a “strategic consultant” and the Sun’s former political editor George Pascoe-Watson as a partner.

Joanna Shields

Chief executive, Tech City; former head of Europe, the Middle East and Africa for Facebook

Joanna Shields, the new chief executive of the Tech City Investment Organisation, has internet pedigree, having worked with Google, Bebo, AOL and Facebook. She may have been unable to save Bebo, one of the social networks caught in the squeeze between the dwindling Myspace and nascent Facebook, but her reputation in the tech world remains strong. Her task now is to transform Tech City into Britain’s version of Silicon Valley.

For years Britain has lagged behind the US in the technology sector. We used to do well; the ZX Spectrum and BBC Micro encouraged a generation of bedroom coders which many credit with launching the British IT industry, and companies such as ARM, Codemasters and Eidos used to be at the top of their game. Yet underinvestment, a university culture that looked down on computer science and a lack of any central location for the community all damaged our lead.

But now the government is staging a comeback, eager to take on Silicon Valley at its own game, and has anointed Silicon Roundabout –the cluster of tech start-ups based around the Old Street area of east London – as the place to do it. The name had to go, though, and so Tech City was born. In the notoriously libertarian world of tech start-ups, the quango was not welcomed as readily as one might have expected. The Register, the IT industry’s house website, attacked it for burning through £1m in just over a year, and others have pointed out that, beyond marketing and PR, the organisation’s main aims – feeding the needs of tech entrepreneurs into No 10’s policy considerations – could be achieved by one person acting as a link between the two.

If Tech City is to achieve its goals, it must overcome a few problems. One element of Silicon Valley’s success was that, until the boom, it was a cheap place to be. Land, housing and the cost of living were all low. That can’t be said for central London. Even in an industry where surviving on Pot Noodle and coding for no pay are marks of pride, it’s a bit much to expect young entrepreneurs to be able to afford the rents in Silicon Roundabout.

On smaller initiatives, though, Tech City’s influence is already showing. The government’s policy on digital matters has greatly improved, as the implementation of the 2011 Hargreaves recommendations on copyright reform demonstrated. Britain now has an intellectual property regime fit for the 21st century, even if it’s more 2000 than 2013. And if Shields finds her hotline to No 10 is more responsive than that of her predecessor, that success may be the first of many.

Howard Davies

Professor, Institut d’Études Politiques, Paris

A former management consultant and civil servant, Howard Davies is one of the ultimate establishment insiders. He has been the controller of the Audit Commission, the first chairman of the Financial Services Authority, director general of the Confederation of British Industry and the deputy governor of the Bank of England. He has worked for both the Treasury and the Foreign Office and served as a trustee of the Tate Gallery and he chaired the 2007 Man Booker Prize judges.

However, Davies is best known for resigning as director of the London School of Economics over the LSE’s links to Muammar al-Gaddafi’s regime. As Libyans battled to overthrow their brutal dictatorship, Davies conceded that the LSE’s reputation had been damaged by accepting £300,000 in research funding from a foundation controlled by Gaddafi’s son Saif. The total amount solicited from the foundation ran to £1.5m. Davies admitted having made a “personal error of judgement” in giving advice to Libya on how to modernise its financial institutions.

His fall has not been painful: he is now a professor at the Institut d’Études Politiques (“Sciences Po”) in Paris and the chairman of the UK Airports Commission, and also sits on a series of institutional boards, including those of Prudential plc and the National Theatre. However, the Gaddafi affair illuminated the way in which Britain’s corporate, public and political institutions work together. The LSE’s accommodation of the Gaddafi family was just one element in a broad attempt to woo the oil-rich Libyan state.

In 2006, Anthony Giddens, another former director of the LSE and the architect of New Labour’s “Third Way”, visited the Libyan capital, Tripoli. In an account of his trip for the New Statesman, Giddens outlined Gaddafi’s theory of “direct democracy” and praised Gaddafi père et fils for “the rehabilitation and potential modernisation of Libya”. The following year Tony Blair met Gaddafi in a Bedouin tent outside Tripoli. With him was Peter Sutherland, the then chairman of BP – and soon to become chair of the LSE’s court of governors. British companies gained access to Libya’s oil reserves; Gaddafi got help from MI6, under the guise of the “war on terror”, in clamping down on dissidents such as Sami al-Saadi, who last year was awarded £2.2m in compensation for Britain’s role in his torture. While the colonel met his grisly end in a sewer pipe, those who did business with him have prospered.

Neil Woodford

Head of UK equities, Invesco Perpetual

In many ways, Neil Woodford is the antithesis of the kind of financier that post-crisis Britain loves to hate. He doesn’t even work in the City – Invesco Perpetual, where he is head of UK investment and presides over funds worth £30bn – is based in Henley, Oxfordshire. There haven’t been any bonus-hunting career moves between Square Mile firms for him; he joined Invesco in 1988 and has been there ever since. To top it off, he didn’t even go to Oxford or Cambridge – this City slicker studied economics and agricultural economics at Exeter.

His influence is undimmed, even augmented, by his background and under-the-radar way of working. Woodford runs both Invesco Perpetual’s income and high-income funds, the latter being one of Britain’s largest investment funds, with shareholdings in a big slice of FTSE-250 companies and assets of £12bn. His portfolios are made up largely of deposits from small investors – pension funds, £50-a-month savers and Isas. An awful lot of people have, in one way or another, put their money in Woodford’s hands, and the decisions he makes have the power to ripple out to millions of UK households.

What he does with their money allows him to pull levers behind the scenes in some of Britain’s biggest companies. His funds have multibillion-pound stakes in the tobacco giants BAT, Reynolds American and Imperial Tobacco, as well as utilities such as the Drax Group (the power company) and BT. Another big interest is BAE Systems: if Angela Merkel hadn’t stepped in to kill off its merger with the Franco-German aerospace giant EADS, Woodford would probably have made the same decision and determined the fate of one of the Ministry of Defence’s largest British suppliers. He is a kingmaker, too – the Guardian reported last year that the word in the City is that it was he who forced out AstraZeneca’s chief executive David Brennan.

Kevin Moses

Director of science funding, Wellcome Trust

The Wellcome Trust is perhaps one of the most prolific grant-awarding bodies in British science and, as its director of science funding, Kevin Moses is in the hot seat. Started in 1936 with money left by the USborn philanthropist Henry Wellcome, the trust’s endowment has grown over the past 77 years to £14.5bn today. Most of the money is spent on charitable grants to researchers and others working in the field of biomedical science. Its work led to the publication of 4,433 scientific papers in 2011 and in the year 2011/2012 it awarded 970 research grants.

The pharmaceutical firms probably control a larger proportion of scientific funding than the Wellcome Trust and the National Institute for Health and Clinical Excellence (NICE) has more say in deciding which medicines come to market in Britain.

The pharmaceutical industry is fixated on hunting for profitable medicines; NICE has a far more routine regulatory job. By contrast, the Wellcome Trust is bound only by internal decisions on where it chooses to focus its efforts. It has no pledge drives, no donors to keep happy and not much of a public image to defend. It can focus its funding where it helps the most, and where it will fill in gaps missed by other bodies.

One hopes that Dr Moses understands that with great power comes great responsibility.

Tony Mitchell

Director, Tesco supply chain

Tony Mitchell is the model of a Tesco company man. He started on the shop floor in 1978 and worked his way up to store manager, then eventually to head office, and now he decides what £1 in every £7 in the UK is spent on.

Getting on to the shelves at Tesco can make a young company, and getting thrown off them can destroy a business. That is something the suppliers of its Everyday Value burgers will be learning to their cost after buying meat from Poland, rather than Britain or Ireland, as their agreement with Tesco stipulated. When it became apparent that the burgers were tainted with horse meat – up to 29 per cent, according to reports – Tesco dropped the supplier altogether.

Although the supermarket’s core business remains groceries, it has a grasp on many other sectors. Take bookselling. Until the Net Book Agreement began to collapse in 1994, books were sold at a fixed price in Britain, allowing independent shops to compete with the chains and ensuring that publications cost the same in all shops. But as competition entered the trade, so the supermarkets used their strength. Now the big three supermarkets are arguably as significant as Amazon.

But where Amazon offers near-infinite selection, the supermarkets restrict what they stock to preserve shelf space and chase economies of scale. As a result, Tesco’s two book buyers were named jointly the twelfth most powerful person in the industry by the Guardian, which argued that they “reflect sales charts but also shape them”. The Tesco story is similar in music and video games.

While the internet offers a long tail to those who want to build slowly, success or failure in the mass market is dictated by an evershrinking group of people such as Mitchell.

Natalie Evans

Director, New Schools Network

Free schools are Michael Gove’s signature policy – a glimpse of what education could be like without “undue interference” from local authorities. The requisite legislation was passed in 2010, but immediately there was a snag: who would have the time, inclination and money to set up a school? Parents’ groups, such as the one led by journalist Toby Young in west London, were in short supply.

Enter the New Schools Network (NSN), whose director is Natalie Evans, a former deputy director of the Conservative Research Department and of Policy Exchange – the think tank whose director Neil O’Brien recently left to work for the Chancellor, George Osborne.

Evans took charge of the NSN at the start of this year, replacing Rachel Wolf, a special adviser to Gove while he was shadow education secretary. Wolf has moved to New York to work for Rupert Murdoch at Amplify, the new education division of News Corp.

The NSN is a registered charity, although it is not clear who its donors are. Its remit is, nebulously, to “support” free school applicants. Most of these have turned out to be faith organisations, education companies or existing sponsors of academies.

The connections between the NSN and the Department for Education are close – sometimes uncomfortably so – and campaigners and opposition MPs such as Lisa Nandy question the organisation’s lack of transparency. For instance, between July and December 2010, the Education Secretary’s confidant Dominic Cummings was employed by the NSN as a paid freelancer. From August to December of that same year, he held one of the four parliamentary passes the minister was allowed to give out, and could come and go from Westminster as he wished.

As the Bureau of Investigative Journalism reported, “In November 2010, while Cummings was freelancing at NSN and enjoying DfE access through his parliamentary pass, the department finalised a grant to the NSN. The grant, for £500,000, was awarded to the organisation in June without being advertised and without inviting any other orga - nisations to tender.” In May that year, Cummings had emailed civil servants urging them to fast-track cash to the NSN, saying: “Labour has handed hundreds of millions to leftie orgs – if u guys cant navigate this thro the bureauc then not a chance of any new schools starting!!”

The close links between the NSN and Gove’s inner circle have led civil servants in the Department for Education to feel that they are being excluded from policy decisions at a time when the government is pushing through sweeping reforms. That suspicion was compounded in 2011 when the Financial Times reported that Gove and his advisers were discussing government business using private email accounts, bypassing Freedom of Information requests.

Meanwhile, Gove’s flagship policy is still struggling to catch on – just 24 free schools opened in 2011, and another 55 in 2012.

Andrew Dilnot

Warden, Nuffield College

How we are to pay for elderly care is one of the great unsolved problems of our time. When the present government came to power in 2010, it turned to Andrew Dilnot to provide that solution. The Dilnot commission’s report – which appeared in July 2011 – received cautious cross-party support, though its implementation is still in doubt. One thing is certain: over the next ten years, it will be impossible to discuss the topic without mentioning Dilnot’s name.

An economist by profession, Dilnot has long occupied a succession of platforms that allow his voice to be heard. He was the director of the Institute for Fiscal Studies between 1991 and 2002, then principal of St Hugh’s College, Oxford, and in 2011 he was appointed Warden of Nuffield College, a graduate research college with an endowment of well over £100m. The college has long had ties to Whitehall and Westminster, and these have only grown closer in the 21st century; many of its fellows are former cabinet members, civil servants and Fleet Street editors.

Last year, Dilnot became the chair of the UK Statistics Authority, and he continues to be engaged with public policy as well as exerting political influence.

Research by Caroline Crampton, George Eaton, Sophie Elmhirst, Alex Hern, Helen Lewis and Daniel Trilling

This article first appeared in the 11 February 2013 issue of the New Statesman, Assange Alone

© MARK PETERSON/REDUX/EYEVINE
Show Hide image

Goodbye to the Confederate flag

After the shootings in Charleston, the Republican right showed it was finally ready to reject the old symbols of the Confederacy.

On 27 June, an African-American activist named Bree Newsome woke up before dawn, put on her climbing equipment and scaled a 30-foot flagpole on the lawn of State House in Columbia, South Carolina. She then removed the Confederate battle flag that flew from it. “We can’t wait any longer,” she explained later in an online statement. “It’s time for a new chapter where we are sincere about dismantling white supremacy.”

After she was led away in handcuffs, the flag was raised again.

Newsome’s protest reflected a growing impatience within America’s black community and anger about liberal inaction. Political rallies by the Democratic presidential contenders Hillary Clinton and Bernie Sanders have been disrupted by the Black Lives Matter campaign against violence committed on young African Americans and the cultural and legal biases that justify it. While promoting his book on race in the US, the writer Ta-Nehisi Coates argued that, to African Americans, the battle flag represents a lingering attempt “to bury the fact that half this country thought it was a good idea to raise an empire rooted in slavery”.

Yet, on this matter, to everyone’s surprise, the black civil rights movement and many southern Republicans have proved to be of one mind. On 9 July the House of Representatives in South Carolina voted to lower the battle flag for good. It stood, representatives said, for racism. It had to go.

The context of this agreement was a painful one. Ten days before Newsome’s act, a 21-year-old white man named Dylann Roof shot and killed nine black worshippers at the Emanuel African Methodist Episcopal Church in Charleston, South Carolina. According to his room-mate, he wanted to start a race war. The TV screens showed a photo of him holding a gun in one hand and a Confederate battle flag in the other.

If the demands for redress made by civil rights groups didn’t come as a surprise, conservative acquiescence did. The Republican Party had built a solid base in the South by courting white voters who cherished the memory of the Confederacy. Yet the party’s presidential hopefuls from both the North and the South – including Jeb Bush, Lindsey Graham, Scott Walker and George Pataki – said that the battle flag ought to be lowered. The most striking intervention was made by the governor of South Carolina, Nikki Haley, who denounced the use of the Confederate flag and signed the bill removing it. Haley is now tipped to figure on the list of potential vice-presidential nominees.

The volte-face of the US right is in part a result of the horror of the Charleston shootings. Yet it also occurs in the context of major shifts within American society. There are still many conservatives who will defend Confederate heritage as a matter of southern pride but the culture wars are changing as the US becomes increasingly European in outlook. This is taking place across the country. It just happens to be more pronounced in the South because no other region has fought so violently and so long to resist the liberal tide.

The story of the battle flag is the story of the South. The first official Confederate flag used in the civil war of 1861-65 caused confusion during fighting – through the haze of gun smoke, its design of 13 stars and red and white bars was hard to distinguish from the Stars and Stripes. An alternative blue cross was rejected for being too sectarian; the racist Confederacy was anxious not to offend its Jewish citizens. So the cross became a diagonal X. This flag was never officially adopted by the Confederate army. In the years after the war its use was infrequent.

There was little need to visualise southern difference in a flag. It was self-evident in the physical signs of racial segregation: separate schools, pools and drinking fountains; black people confined to the back of the bus. Political displays of the battle flag of Dixie (the historical nickname for the states that seceded from the Union) only really resurfaced when that racial order was challenged by northern liberals. In 1948, the Democrats – then the party overwhelmingly in control of the South – split over modest calls for civil rights. The conservatives who refused to support that year’s presidential ticket, the “Dixiecrats”, triggered a rev­ival of flag-waving across the region.

The old battle flag suddenly appeared on private lawns, on cars and at political rallies. Supposedly ancient cultural traditions were invented overnight. For instance, the 1948 student handbook of the University of Mississippi confessed: “Many Ole Miss customs are fairly new; they lack only the savouring which time brings . . . Ole Miss has adopted the Confederate flag as a symbol of the Mississippi spirit. Each football game finds the scarlet flag frantically waving to the rhythm of the Rebel band.”

I can confirm that this “tradition” was still going as recently as in 2005. That year, I attended an American football game at Ole Miss and was surprised when the band played “Dixie” at the end. White boys and white girls stood up and belted out the folk song of the Confederacy, while black students filed out.

In 1958, South Carolina made it a crime to desecrate the battle flag. Three years later, on the 100th anniversary of the outbreak of the civil war, it was hoisted above its Capitol building in Columbia. That day, there was a struggle in the US Congress to keep federal funding going for segregated schools.

So clear is the link between the postwar white resistance to civil rights and the battle flag that many see it as the symbolic equivalent of the N-word. Jack Hunter, the editor of the conservative website Rare Politics, says: “Some people insist that it’s not about racism, not about slavery, not about segregation. But it’s about all those things.” Hunter grew up in Charleston and used to skateboard in the car park of the church that Dylann Roof attacked. When he was a young journalist, he appeared on local radio as a rabidly right-wing masked character called “the Southern Avenger”. His past was exposed in 2013 while he was working for Rand Paul, a Republican presidential candidate, and Hunter stepped down from his position. He publicly renounced his youthful association with racial conservatism. He now eschews any romanticism about the Confederate cause and its demand for states’ rights. “States’ rights to do what?” he asks: the right to discriminate against African Americans? He is glad that the State House flag is gone. He ascribes its longevity to ignorance, which was corrected by Roof’s rampage: “It was the first time that [southern Republicans] were able to see a different perspective on this symbol.”

Not everyone agrees. Richard Hines – a former South Carolina legislator, Reagan campaign state co-chair and senior activist with the Sons of Confederate Veterans – insists that the flag is “an enduring symbol of the southern fighting man”. Indeed, a poll in July found that 57 per cent of Americans think it stands for southern heritage, rather than racism. Yet that heritage has a political dimension. “Southern people are proud of who they are and there is a leftist assault to destroy the best part of America,” Hines says. “The Trotskyite elite in control of the establishment wants to root out the southern tradition” – a tradition of religious devotion, chivalry and military honour. It is possible to cast the battle flag as a pawn in a much larger cultural conflict.

In 2000, civil rights activists lobbied hard to get the battle flag removed from the top of the South Carolina Capitol and succeeded in having it shrunk in size and relocated to the grounds of State House. The issue came up in that year’s Republican presidential primaries – an unusually poisonous contest between George W Bush and John McCain. Supporters of Bush put out a false story that McCain had fathered an interracial child out of wedlock. McCain added to his woes by opining that the battle flag was “a symbol of racism and slavery”. An organisation called Keep It Flying flooded the state with 250,000 letters attacking him and he lost the crucial competition here to Bush.

The battle flag has retained a strong emotional power for a long time. This makes the Republican establishment’s abandonment of the flag all the more surprising. Then again, those who run the South are probably the people most likely to grasp how much the region has changed in just a decade.

***

In 2010 I took a trip through North Carolina. The landscape told a story. Dotted along the roadside were abandoned black buildings, the old tobacco sheds. The decline of the rural economy had rendered them obsolete. Over the fields that would once have been full of farmers were freshly tarmacked roads, stretching out to nowhere. My guide explained that these were supposed to be cul-de-sacs for new houses. North Carolina was going through a property boom. But who was going to buy all those homes, I asked? The answer: damn Yankees.

Demography is destiny. This once agri­cultural region developed fast from the 1960s onwards by keeping union membership, taxes and regulation as low as possible. Yet capitalism proved disastrous for southern conservatism. Northerners flooded in, seeking work or retirement and bringing their own values. The forecast is that North Carolina’s Research Triangle – the South’s Silicon Valley – will grow by 700,000 jobs and 1.2 million people in two decades.

White migration was accompanied by an influx of Spanish speakers as the service sector flourished. Between 2000 and 2010, the white share of the population of North Carolina fell from 70 to 65 per cent. The black proportion remained at roughly 21 per cent. The Latino proportion, however, jumped from 4.7 per cent to 8.4 per cent. Today, the proportion of people who are non-white and over 60 is about a third. But it’s approaching nearly half for those under 18. As a result, politics in the South is no longer biracial: a contest between white and black. It is increasingly multiracial and uncoupled from the region’s complex past.

The impact of these changes is reflected in voting patterns. In 2000, the South was still overwhelmingly Republican in presidential contests. Even the Democratic nominee, Al Gore, a southerner, lost his home state of Tennessee. But in 2008 and 2012, Barack Obama took those states with the fastest-changing demographics: Florida and Virginia. He won North Carolina in 2008 and lost it in 2012 – but by less than 100,000 votes. It is true that the Republicans won back control in the 2014 midterm elections, with the result that the Deep South now sends few Democrats to Congress; but the region’s political masters are not quite as traditional-minded as they once were.

The Republican relationship with the Confederate past is complex. As the party of Abraham Lincoln and the Union, the GOPs’ southern support was historically small. But in the 1960s the national Democratic Party embraced civil rights and alienated its once loyal southern following; the Republicans took the opportunity to steal some conservative white voters.

The growing southern Republican vote had a class component. Its success in local and congressional races was built more on winning over middle-class moderates than on appealing to the working-class racists who filled the ranks of the Ku Klux Klan. The southern Republican Party did enthusiastically embrace the Confederate battle flag in many quarters. But some office-holders did so only with ambiguity, while large sections of the party never identified with it at all. The period of Republican ascendancy in the South was, in reality, linked with a softening of the area’s racial politics.

Two of the Republicans’ current southern stars are Indian Americans: Bobby Jindal, the governor of Louisiana, and Nikki Haley, the anti-flag governor of South Carolina. There are just two black people in the US Senate and one of them is a Republican, the Tea Party-backed senator for South Carolina, Tim Scott. Marco Rubio, the Floridian senator and presidential candidate, is Cuban American, and the former Florida governor Jeb Bush is married to a Mexican-born woman and speaks fluent Spanish. Bush has tried to push a more moderate line on immigration, in deference to how the GOP will struggle to win the White House if it appeals only to angry white voters. The Kentucky libertarian senator Rand Paul, Jack Hunter’s former boss, has called for legal reforms to correct the trend of keeping far more black than white people in prison. And he is not the only Republican to have been moved by recent race riots sparked by police violence.

***

Violence on the streets of Ferguson, Missouri, and Baltimore, Maryland, confirmed that there still is a culture war in the US. Yet its character has changed. In the past, civil disturbances were typically leapt upon by conservative politicians as evidence of social decline. The 1992 LA riots were blamed on single parenthood and rap lyrics. In contrast, conservative leaders today are far more likely to acknowledge the problems of white racism. There is no place in their ranks for the likes of Dylann Roof. White supremacists are tiny in number.

Jack Hunter claims: “The KKK is like 12 guys in a telephone booth. Liberal groups will use their threat for fundraising but it doesn’t exist. It hasn’t properly since the 1960s.” Roof’s actions say more about gun control, mental illness and the angst of the young than they do about popular, largely liberal views on race, as polling shows.

We can see a similar liberal shift in other areas of the historic culture war. In May 2015 Gallup released the results of a “moral acceptability” survey charting changes in national attitude across all age groups, from 2001 to 2015. Approval of gay relationships jumped from 40 to 63 per cent; having a baby out of wedlock from 45 to 61 per cent; sex between unmarried men and women from 53 to 68 per cent; doctor-assisted suicide from 49 to 56 per cent; even polygamy went from 7 to 16 per cent. Abortion remained narrowly disapproved of: support for access has only crept up from 42 to 45 per cent. This is probably a result of an unusual concentration of political and religious opposition and because it involves a potential life-or-death decision. But the general trend is that young people just don’t care as much about what consenting adults get up to.

Why? It might be because old forms of identity are dying. One way of measuring that is religious affiliation. From 2007 to 2014, according to Pew Research, the proportion of Americans describing themselves as Christian fell from 78 to 71 per cent. Today, only a quarter of the population is evangelical and 21 per cent Catholic, down despite high immigration. Then there is the decline in civic or communal activity. Since 2012, the organisers of Nascar, the stock-car races, have not published attendance figures at their tracks, probably because they have fallen so sharply. The decline of this most macho and working class of sports parallels the fall in conservative forms of collective identity such as southern traditionalism.

The old culture war was, like the racial politics of the old South, binary. In the 1950s, around the same time as the South invented its tradition of flying the battle flag in colleges, the US constructed an ideal of the “normal” nuclear family unit: straight, white, patriarchal, religious. On the other side was the “abnormal”: gay, black, feminist, atheist, and the rest. The surest way to get elected in the US between 1952 and 2004 was to associate yourself with the economic needs and cultural prejudices of the majority. The approach was once summed up by a Richard Nixon strategist thus: split the country in two and the Republicans will take the larger half. But that is changing. The old normal is no longer the cultural standard but just one of many identities to choose from. The races are mixing. Women want to work more and have children later in life, possibly without marriage. Many religious people are having to rethink their theology when a child comes out as gay. And the enforcers of the old ways – the unions, churches or political parties – are far less attractive than the atomising internet.

***

Politicians are scrabbling to keep up with the diffusion of American identity. Democrats got lucky when they nominated Barack Obama and chose a presidential candidate who reflected the fractured era well: interracial, non-denominational Christian, and so on. In the 2012 presidential race the Republicans got burned when they tried to play the old culture war card on abortion. They won’t repeat that mistake. After the Supreme Court legalised gay marriage across the country in June, the right’s response was not as uniformly loud and outraged as it would have been in the past. Some protested, but serious presidential contenders such as Jeb Bush grasped the implications of the defeat. There is a cultural and political realignment going on and no one is sure where it will lead. It’s encouraging caution among the Republican top brass. It is time, they think, to abandon lost causes.

The death of southern traditionalism is part of the ebb and flow of cultural history. Identities flourish and die. As political fashions change, you find the typically American mix of triumph on one side and jeremiad on the other. Richard Hines stood vigil as the battle flag was lowered in Columbia and noted with disgust the presence of what he described as “bussed-in” activists. “They pulled out all these gay pride flags and started shouting, ‘USA, USA, USA!’ It reminded me of the Bolshevik Revolution.”

Hines reckons that more southerners will now fly the flag than ever before and says he has attended overflow rallies of ordinary folks who love their region. He may well be correct. The faithful will keep the old Confederate standard fluttering on their lawns – an act of secession from the 21st century. But in the public domain, the battle flag is on its way down and in its place will be raised the standard of the new America. The rainbow flag flutters high. For now.

Tim Stanley is a historian and a columnist for the Telegraph

This article first appeared in the 20 August 2015 issue of the New Statesman, Corbyn wars