Show Hide image

How the miners’ strike of 1984-85 changed Britain for ever

Thirty years later questions still haunt many of those who took part. Could the outcome have been different?

Murton Colliery in County Durham, 1992. The pit ceased production the previous November. Photograph: Bob Anderson/Rex Features

One recent grey afternoon I visit the site of the old Shirebrook colliery in north-east Derbyshire with a former miner, Alan Gascoyne. “This was where we picketed . . . that was the union office, there were the shafts, and that was the coal preparation plant.” We are looking through his car windscreen at a huge steel warehouse containing Sports Direct’s central “distribution unit”.

Sports Direct is owned, like Newcastle United Football Club, by the billionaire Mike Ashley. “I’ve never met him but he comes here by helicopter,” Gascoyne says. The unit is a very 21st-century operation; its retail staff are mainly on zero-hours contracts, and most of the 5,000 people working at the warehouse are supplied by two big manpower agencies. On New Year’s Day, a newborn baby was found abandoned in a lavatory; the young mother, thought to be from eastern Europe, was arrested but later released. Employees complained to the local paper about a “strikes” policy under which six minor infringements – such as lateness or failure to meet targets – were punished with summary dismissal. The young mother was rumoured to have been close to six strikes and threatened with the sack if she took time off, late in her pregnancy. Of the colliery, which closed in 1993 and was replaced 12 years later by the warehouse as the town’s largest employer, there is no trace.

But the mine is not forgotten. Gascoyne, a forceful 59-year-old ex-faceworker who swears a lot, was the branch secretary of the National Union of Mineworkers throughout the strike of 1984-85 and until the pit closed. The son of a communist miner, he has kept its memory alive at the Shirebrook Miners’ Welfare Social Club, founded in 1920. The bar and the hire of its large hall help to finance the local Alzheimer’s group, OAP dance classes, a disco for the disabled and Outward Bound courses for the primary school. The bar and adjoining rooms are heavily decorated with mementos, including a rare photograph of a shirtsleeved A J Cook, the miners’ leader in the great lockout of 1926, inscribed in copperplate with lines from Idris Davies’s poetic chronicle of that heroic defeat, “The Angry Summer”:

Here is Arthur Cook, a red rose in his lapel
Astride on a wall, arousing his people . . .
And tomorrow in all the hostile papers
there will be sneers at Cook and all his capers
and cowardly scribblers will be busy tonight
besmirching a warrior with the mud of their spite.

There wasn’t another such “angry summer” (or as “besmirched” a miners’ leader) until 1984. In the pivotal episode of Margaret Thatcher’s premiership, the families of “the best men in the world, who beat the Kaiser’s army and beat Hitler’s army” – in the words of the 90-year-old Harold Macmillan’s agonised speech to the House of Lords in November 1984 – endured extreme hardship, hunger, lack of fuel and police harassment. In the process, colliery communities powerfully refreshed their traditions of collective self-help. Women, in a notably male-dominated society, not only formed a transformative and unpredicted network of support groups, raising funds to feed and sustain the most financially pressed in almost every mining village, but asserted themselves politically, travelling across the country to make the miners’ case, appeal for support and join the picket lines.

Yet that great struggle ended, like the 1926 lockout, in failure. Thirty years later two questions still haunt many of those who took part. Could the outcome have been different? And what were its long-term consequences?

On the first, those who with hindsight see the 1985 defeat as inevitable point to the unprecedentedly comprehensive arm­oury of the state mobilised by Thatcher. Her biographer Charles Moore records how she summoned Willie Whitelaw immediately on taking office in 1979 and announced: “The last Conservative government was destroyed by the miners’ strike. We’ll have another and we’ll win.” Thatcher sought not only to avenge the miners’ victories of 1972 and 1974, but also to erase the mystique attaching to the NUM, its solidarity reinforced by the small and cohesive communities in which miners lived, by the mutual dependence required by the dangers underground, and by widespread public admiration.

Moore says that the Whitehall Civil Contingencies unit in 1979 thought that “if there was to be a strike, it should begin in the spring and . . . be over pit closures, which tended to divide the union, rather than over pay, which tended to unite them”.

These two conditions were eventually fulfilled, though not until three years after an attempted pit closure programme ended in a climbdown. The government had not yet implemented Nicholas Ridley’s infamous 1977 confidential note on how to defeat the miners. This included stockpiling of coal, increasing power-station capacity to switch from coal to oil, use of non-union haulage drivers and cuts in supplementary benefit to strikers’ families. But the government was determined to be ready next time.

In her notorious characterisation of the “enemy within”, Thatcher did less than some of Arthur Scargill’s more fastidious critics to distinguish between the NUM president and its members. Unsurprisingly, given that while she repeatedly portrayed Scargill as a revolutionary Marxist seeking to bring down an elected government (a depiction that Scargill did little to disavow) she was also determined to break what she saw as the NUM’s unhealthily corporatist relationship with the National Coal Board. Dating back to nationalisation in 1946, the relationship between the union and the NCB not only contradicted her view of how an industry should be run, but also symbolised the Attlee postwar consensus she was intent on overturning.

To listen to the ex-miners coming through Gascoyne’s office today for help with social security and pneumoconiosis claims is to be aware of a historic level of union protection that would be incomprehensible to the unorganised Sports Direct workforce. A genial man called Graham Elliot recalls his first lesson in the strength of the NUM branch. In 1972, he and another 17-year-old apprentice fitter, under indentures that explicitly precluded strike action, were walking to the pit baths when they were confronted by the branch secretary. “And he said: ‘Now where are you going, lads? We’re on strike, starting today.’ And we said, ‘Yes, but they’ve told us if we go on strike that’ll end our apprenticeship.’ And he pointed up [to the winding gear]. ‘I’ll tell you summat now, son. If you lose your apprenticeship, those wheels won’t turn until you get your jobs back.’ ”

Gascoyne insists that he co-operated with the NCB in raising production, occasionally by stamping out sectional strikes. “If you’re not producing owt, you can’t make money. To me it were important the pit was a success.”

The secret plan – denied during the strike but admitted by Thatcher in her memoirs – of Ian MacGregor, imported from the United States and appointed NCB chairman in 1983, had been to cut 75,000 jobs over three years. This was slightly more than Scargill was later ridiculed for predicting.

On 1 March 1984, the South Yorkshire NCB announced plans to close Cortonwood Colliery, a challenge to the NUM’s policy of opposing all pit closures on any grounds other than geological exhaustion. With this, and MacGregor’s admission five days later that he envisaged closing, on “economic” grounds, about 20 pits over the next year at a cost of 20,000 jobs, strikes in Yorkshire and Scotland spread to 81 of the country’s 163 pits by 12 March.

At the Shirebrook Miners’ Welfare Social Club, Gordon Butler, NUM area secretary for North Derbyshire, urged a packed and noisy meeting of miners to strike. “The young ’uns were all for striking,” Gascoyne says, “but others were shouting about a [national strike] ballot.”

The status as the “barometer” British coalfield of North Derbyshire, historically led by the left-wing of the two NUM factions, is partly a function of geography. Shirebrook, its biggest pit, was in a north-eastern wedge between South Yorkshire and Nottinghamshire, the area always least likely to strike. The mass meeting was all the stormier because a North Derbyshire ballot had already evenly split the area with a mere 16-vote majority against a strike. But with a militant, trusted local leadership under Gascoyne warning that it would be better to walk out than be “picketed out” by Yorkshire miners, Butler’s call succeeded. Later Shirebrook became so divided that a Metropolitan Police officer called it “the Belfast of England”. But, for now, a large majority of its 1,995 miners went on strike.
 

****
 

To Nicholas Ridley’s prescription, Thatcher had added the appointment of Peter Walker as energy secretary. He was a Tory “wet”, a skilled communicator and ambitious. A clever choice. Working miners and coal-dependent businesses were encouraged to use the government’s new employment laws, which resulted in the sequestration of NUM funds. But it was another Ridley proposal, for “large mobile squads” of police, that would prove crucial. Besides their often huge presence at pitheads and other picketing targets, “mobile squads” were deployed on roads across the country to halt pickets seeking to prevent the movement of coal and would-be working miners.

In the book Settling Scores, Nicholas Jones, then the BBC’s outstanding labour correspondent, reviews cabinet documents that show how a frustrated Thatcher, told by Leon Brittan, the home secretary, that he had gone to “the limit of what [he] could do while respecting the constitutional independence of police forces”, urged him to “stiffen the resolve” of chief constables to adopt “the more vigorous interpretation of their duties which was being sought”. Keeping the peace was subordinated to the paramilitary role of forcing a way through for those breaking the strike – even in South Wales, where the walkout was most solid.

On a moonlit December night in 1984, I joined a group of NUM pickets who were avoiding the checkpoints in a clandestine cross-country yomp up Mynydd Merthyr, a wooded ridge above the village of Aberfan, where just two miners were going to work at Merthyr Vale colliery. The background to the mission was sombre because one of the two miners, David Williams, had been in a taxi a fortnight earlier when two strikers dropped a concrete post from a footbridge on to the car, killing the driver, David Wilkie. I remember how, early the next morning, the police charged on the pickets as the working miners’ new taxi came into view. They trampled my photographer colleague Hugh Alexander to the ground, cracking several of his ribs.

But this was trifling beside the scores of incidents across Britain in which truncheon-wielding policemen – often in riot gear, and making numerous arrests – confronted and diverted pickets. The biggest set-piece battle, now the subject of a “scoping” exercise by the Independent Police Complaints Commission (IPCC) to determine whether there should be a full investigation, was at the British Steel coking works at Orgreave, South Yorkshire, in June 1984.

About 10,000 NUM pickets (Scargill among them), seeking to halt the transfer of coal into the plant and coke out of it, were allowed to assemble – suggesting a set-up – and were then forced back by at least 5,000 police officers, using horseback baton charges and snatch squads. It was a payback for Scargill’s greatest triumph, the Saltley coke works closure in Birmingham during the 1972 strike. But the payback came at a severe cost. The prosecutions against 93 arrested pickets collapsed because the police evidence was unreliable. South Yorkshire Police, sued for malicious prosecution and assault, later paid £425,000 compensation and £100,000 in legal costs to 39 pickets in an out-of-court settlement. The force referred itself to the IPCC in 2012 after a BBC Yorkshire Inside Out documentary discovered identical wording in 34 separate statements by police, triggering an energetic campaign for a full public inquiry.

At Shirebrook, the police were crucial in securing a gradual return to work. This did not reach critical mass until November, by which time the NCB North Derbyshire area director, Ken Moses, had mounted a systematic effort to bus miners back.
 

****
 

To help me “understand” how it had been, Alan Gascoyne gave me a DVD of a June 1984 ITN film made in the village. It shows fighting between pickets and police escorting working miners. From the pavement, women and children shout “Scab”; working miners’ wives complain about intimidation: smashed windows at their homes, bullying of their children at school; a striker is hospitalised for several days after an attack by a working miner. The young, long-haired Gascoyne tells ITN: “We had a meeting – there were 700 there – and we decided that our members wouldn’t work with people who had crossed picket lines.”

It didn’t turn out like that. Gascoyne still despises those who worked from the start of the strike. But he took a different view of those who returned from November 1984 onwards – by which time Scargill’s promise of dwindling coal stocks and “General Winter” coming to their rescue was, Gascoyne says, “all bollocks”. By February 2005, 70 per cent of Shirebrook miners, many of them desperate, had gone back to work.

There is a moment in the film Billy Elliot when Jackie, a loyal NUM member, crosses the picket line to fund his son’s dream of becoming a ballet dancer. Something similar was happening here. “I had a guy come to me at Christmas, a face chargeman, in tears. He was losing his house. His wife was going to go, he’d got all these debts. So I said: ‘We’ve lost, kid, so get back to work.’ ”

Gascoyne, on strike until the end, was re-elected the following July. While his hatred of Thatcher is undimmed – “I wish she’d been burned at the stake” – he retains a certain respect for the late Ken Moses, whom he confronted at a secret night-time meeting convened during the strike by Stephen Verney, the then bishop of Repton, who had served undercover with Greek guerrillas in Nazi-occupied Crete and wanted to hear “both sides of the argument”.

Gascoyne had come from the trial of six young striking miners (who like most of the thousands arrested had never been in trouble with the police before), jailed for setting fire to empty buses used to bring working miners to the pit.

“Moses said: ‘They’ll never work again, them lads. We don’t want arsonists.’

“So I said, ‘Bishop Stephen, he thinks he’s higher than your boss, the Lord. Society says these lads are going to prison but he’s not satisfied with that.’ So the bishop, he was nice and quietly spoken, says: ‘Kenneth, I think you should reconsider that.’ ”

Which Moses promised he would – and eventually agree three years later that the men could be “set back on” at Shirebrook. Unlike MacGregor, Moses had started out at the coalface.

South Yorkshire Police grapple with the pickets outside the Doncaster NCB headquarters, March 1984. Photograph: AFP/Getty Images

Could another strategy have produced a different outcome? Gascoyne doesn’t think a national ballot would have helped the miners, and argues that Nottinghamshire still would not have joined the strike. But there is a strong counter-case. A successful ballot would have afforded the last chance to bring Nottingham out, and so increased support from other unionists who were wary of risking their own jobs when even the miners were continuing to work in the second-largest coalfield. The NUM leadership’s argument against a ballot was that a man in one area did not have the right to vote a man in another out of a job – which could happen if the members rejected a strike, as the NUM leadership reasoned they might.

After all, a year earlier, in 1983, a national ballot had heavily rejected a strike against the closure of Lewis Merthyr Colliery in the Rhondda Valley. Unlike at Shirebrook, the old Lewis Merthyr mineworkings have been preserved in the Rhondda Heritage Park. You can see the winding house with its 1890 steam engine, visit the lamp-room, put on a miner’s helmet and descend to the pit bottom in the cage. Above ground, an elegiac printed notice describes the “two most dramatic episodes” in the pit’s history – a 1956 gas explosion that killed nine men, and a three-day underground sit-in protest by 27 miners in 1983, emerging to a “tumultuous heroes’ welcome”. The notice also says that pits “continued to be closed . . . until a . . . strike by the NUM in 1984-85 ended in bitterness and failure. Yet what else could the miners have done?”

A smaller metal plaque commemorates the official inauguration of the heritage park after the 1987 general election by the minister who, despite his triumphs in the strike, had failed to win promotion: “Peter Walker, Secretary of State for Wales”. The man who had helped to turn the industry into a museum piece got to open the museum.

But mining at Lewis Merthyr was especially difficult and dangerous, a legacy of Victorian coal-owners who plundered the best seams with scant regard for sound engineering principles. Peter Heathfield, then still the North Derbyshire area secretary, told me that his members were saying they would not work in the pit, let alone strike to keep it open.

So the No vote on Lewis Merthyr was not necessarily a persuasive precedent. Two MORI polls of miners – in March and April 1984 – showed majorities of over 60 per cent in favour of a strike. Accepting that a ballot might well have been won, Ken Capstick, an NUM ally of Scargill’s, argued after the strike that organising a vote would have seemed “like giving in”, adding: “You’ve got to remember that we had lads . . . who were picketing every day, total commitment. And they were saying, ‘Shove your ballot up your arse.’ They looked at it that Margaret Thatcher wanted a ballot, Ian MacGregor wanted a ballot, the media wanted a ballot, and they weren’t going to have one.”

But while it was assumed that the government was confident that a ballot would be lost, it is now clear from the cabinet papers that ministers were anything but.

Another criticism, levelled with hindsight by some prominent NUM left-wingers, was over mass picketing: that the highly publicised violence it generated came at the expense of the task of building wider public support for the case for coal. Early in the strike, moreover, NUM South Wales leaders agreed with the steel unions that enough coal would be allowed into the Llanwern steelworks to keep the blast furnaces working. Scargill outlawed such “sweetheart deals” and insisted on picketing to try to stop all supplies to the steelworks. The result was up to 150 trucks loaded with coal and iron ore trundling daily up the M4 from Port Talbot and, thanks to a huge police presence, in to Llanwern.
 

****
 

“It’s all down to one man?” the Queen asked the Times’s labour editor, Paul Routledge, on a visit to the paper late in the strike. Routledge answered that he did not think one man could bring 100,000 men out on strike; but later wrote that he “owed the Queen an apology” and that “by this time” it was down to one man. He was right both times; many tens of thousands of miners were not “brought out” by Scargill but struck to save the industry, or out of loyalty to their local leaders, or because not crossing a picket line was in the DNA, or all of these. But Routledge was also right that “one word” from him could have ended the strike before the scarcely believable hardship of the final months. “And that’s really why people turned against Arthur as a leader,” Gascoyne says now. “There’s a time when enough is enough.”

The last hope, at least of compromise, had passed in late October 1984, when a deal was reached with the pit deputies’ union Nacods (the National Association of Colliery Overmen, Deputies and Shotfirers) after it threatened an all-out strike. Because of its members’ statutory safety responsibilities, this would have shut down the Nottinghamshire coalfield.

At a 30th-anniversary conference at King’s College London this year, Neil Kinnock, still scarred by the torments inflicted on him by the strike, and John Monks, at the time head of the Congress House industrial relations department, described their efforts through the summer and early autumn of 1984 to persuade Scargill to overcome his insistence that no pit could close on economic grounds. Much of this discussion revolved around a proposal that pits would be subject to closure if they could not be “beneficially” (the word was used deliberately instead of “economically”) mined. This was a masterpiece of creative ambiguity which TUC officials thought would allow the NUM president to claim victory, on the grounds that it took into account a pit’s social value as well as its mere profitability. Certainly a hawkish Times leader at the time complained about exactly that, arguing that “beneficial” was not an “innocent word”.

Finally, the TUC’s efforts to leverage the Nacods strike threat into a settlement with the NUM foundered when Scargill summarily rejected the deal accepted by Nacods. This would have provided for any planned pit closure to go before an independent, if non-binding, review body. That review procedure did not save a single pit. But TUC officials believed that with the NUM’s (and their own) authority behind it, it would have had much greater traction. Either way, with the Nacods danger passed, MacGregor and the government were now intent on all-out victory.

Scargill was right in foreseeing the industry’s ultimate destruction. Gascoyne says: “They hate miners, Tories. Always have done. Always will do.” He believes the last hope for the industry evaporated with Labour’s defeat in the 1992 general election – and is convinced that the reduction of the 160 UK deep mines in 1984 to just three today (and a single one by next year), with the loss of 142,000 jobs, “will come back to haunt” the Conservatives.
 

****
 

Deeply unfashionable because of carbon emissions, coal is nevertheless still the single fuel most commonly used (41 per cent) to generate electricity in Britain. It’s hard for ex-miners to accept that today 90 per cent of our power-station coal is imported, the largest single source being Russia (38 per cent). David Cameron, for geopolitical reasons flowing from the Ukraine crisis, is now arguing that Britain should be less dependent on Russia, and should therefore pursue the unproven domestic alternative of shale gas extracted through fracking.

The cost argument prevailed, though it’s a moot point whether the market intervention required for large-scale investment in the most potentially productive Midlands and Yorkshire coalfields, coupled with the development of clean coal technology, would have been any greater than the present one for nuclear power, with EDF now guaranteed twice the standard wholesale electricity price for building the Hinkley Point C nuclear plant. At the very least, the British coal industry could have been obliged – as its counterpart in much more “corporatist” and co-determinist Germany has done – to diversify into other industries employing ex-miners, and retraining them before they become redundant, with much less damage to pit communities.

So what were the legacies beyond the coal industry? One arose from the aggressive policing of the strike and was woefully under-reported at the time.

Settling Scores makes a persuasive case on what its editor, Granville Williams, calls the “many parallels” between the role of the “media, South Yorkshire Police and politicians” over Hillsborough and the steelworks battle five years earlier. Indeed, it is hard to challenge the belief of many former miners that, without Orgreave, the lethal malfeasance – and subsequent cover-up – by South Yorkshire Police at Hillsborough would not have happened. The Thatcher government’s elevation of the police above the norms of local accountability, not to mention the law, surely helped to instil in them a lasting sense of impunity, which is only, slowly, unravelling today.

In her memoirs Thatcher summed up how she saw the strike’s outcome:

From 1972 to 1985 the conventional wisdom was that Britain could only be governed with the consent of the trade unions . . . Even as we were reforming trade union law and overcoming lesser disputes . . . many on the left and outside it continued to believe that the miners had the ultimate veto and would one day use it. That day had now come and gone.

While Scargill’s most devoted supporters rejected it, Thatcher’s boast was not seriously challenged elsewhere, not even by several in the NUM’s own Labour-left/communist caucus – any more than was her confident belief that she had seen off the consensus that had prevailed, however modified by Tory governments since Attlee. The union was not destroyed when the miners went back without an agreement in March 1985. Thatcher continued to think a second national strike over closures possible. Yet it is hard now to see how the union could have commanded support for one. Terry Thomas, the left-wing NUM vice-president in South Wales during the strike and a supporter of Scargill’s presidential campaign in 1982, successfully proposed the return to work resolution at the special conference in March. He told me 30 years later at his home in Gowerton, near Swansea:

The men returned to work not because they had stopped believing in what they were fighting for . . . Houses were being repossessed, marriages were breaking up, the kids were going without, and there was no end in sight. We were not now picketing steelworkers to try and stop coal or steel. We were picketing people who had stood on the picket lines with us for a whole year. Proud, strong miners crying because they were going back to work. We had no right to demand that they continue with the strike.

Thomas sees it as a “defeat for the whole working class, a defeat for us all”, without which the battery of exploitative employment practices, such as zero-hours contracts, would not have been possible a generation later. “If ever the trade union movement needed a victory which could prove to people that they need to protect their rights, it’s now.”

It is certainly impossible to separate yawning income inequality from the halving of union membership since its 1979 peak. By ending the syndicalist dream, the defeat “clarified” for the left, to the eventual benefit of a transformed Labour Party, that the ballot box was the only way to effect political change in Britain. But something else, more social democratic than syndicalist, got lost, too. Because the NUM was the trade unions’ praetorian guard, the defeat robbed organised labour of an important source of its self-confidence. It hastened the erosion of the trade unions’ role in a political settlement that had lasted not only since 1972, or even 1945, but – for all its ups and downs – since Edwardian times.

In saying that the NUM had “blown its industrial strength”, David Basnett, the then leader of the National Union of General and Municipal Workers, had in mind not only the coal industry but the TUC’s wider authority, underpinned by the power of its strongest affiliate. The National Economic Development Council and the Manpower Services Commission (set up by the Macmillan and Heath governments), on which the unions had sat on equal terms with the employers, were just two casualties.

For Thatcher, and probably for Scargill, these tripartite bodies were undesirably corporatist. But they provided a platform from which to challenge, or at least counterbalance, the increasing influence of a deregulated City of London, to lend institutional respectability to protests at the grotesque distortion, recently depicted by Oxfam, that today leaves five families, including Mike Ashley’s, owning more in wealth and assets than the 20 per cent of Britons – 12.6 million people – who are worst-off.

Much of Margaret Thatcher’s new neoliberal settlement, replacing the old social-democratic one, survived and indeed helped to shape New Labour. As she saw it, the defeat of the miners’ strike of 1984-85 cut the brake on the application of largely unrestrained market forces, not only in energy but in the wider ordering of society. At a time, post-2008 crash, when it is no longer so taboo to question the efficacy of unregula­ted markets, that is something to mourn.

This article first appeared in the 11 June 2014 issue of the New Statesman, The last World Cup

Martin O’Neil for New Statesman
Show Hide image

Why the British addiction to period drama is driving away our best black and Asian actors

There is a diversity crisis in British TV and film as, increasingly, stars are decamping to America to make their career there.

Back in April, a six-part drama called Undercover premiered on BBC1. Perhaps you were one of the five million people who watched it: the story was audacious and continent-hopping, enfolding a narrative about a man on death row in the United States with an all-too-believable tale of a Metropolitan Police officer who marries a woman he is meant to be keeping under surveillance.

The reason the programme attracted so much attention, however, was not what it was about, but whom. Starring Sophie Okonedo and Adrian Lester, Undercover was widely reported as the first mainstream British television drama with black actors in the lead roles. This wasn’t true: as James Cooray Smith wrote on the New Statesman website, that milestone was passed in June 1956 by Mrs Patterson, a BBC adaptation of a Broadway play starring Eartha Kitt.

Yet Undercover was still a breakthrough. Smith, casting his mind back over more than six decades of British television, could not think of more than a handful of other examples. Writing in the Observer, Chitra Ramaswamy expressed her feelings with quiet devastation: “In 2016, it is an outrage that it’s a big deal to see a successful, affluent, complicated black family sit at a ­dinner table eating pasta.” Think about that. In 2016 in Britain, a country where more than nine million people describe themselves as non-white, it is news that a black, middle-class family should not only feature in a prime-time BBC drama but be at its heart. Undercover exposed how white most British television is.

Actors of colour have appeared on British film and TV screens for decades, and they have been visible on British stages for centuries – yet they have been shunted into the margins with depressing regularity. In January the actor Idris Elba urged British MPs to take the matter seriously. “Although there’s a lot of reality TV,” he argued, “TV hasn’t caught up with reality.”

In February, there was renewed uproar over the lack of racial diversity in Hollywood at the 88th Academy Awards, and the infuriated hashtag #OscarsSoWhite blossomed again on social media. A month later, Lenny Henry argued that black and minority ethnic (BAME) talent was being “ghettoised”. The term could hardly be more charged. Speaking at the London premiere of Mira Nair’s film Queen of Katwe, the actor David Oyelowo said: “What we need now is for a change to come. I think the talk is done.”

There has been some change. In March, the Royal Shakespeare Company opened a production of Hamlet starring Paapa Essiedu, an actor of Ghanaian heritage raised in London. It was the first time that a black performer had taken the role for the company. A new set of BBC diversity targets both on- and off-screen was unveiled in April. Noma Dumezweni is playing Hermione in Harry Potter and the Cursed Child in the West End, and in October the BFI launched Black Star, a nationwide season celebrating black talent in film and TV. But what does the picture really look like, in late 2016? And what, if anything, needs to change?

The first challenge is that many in the film and TV industry find it difficult to talk about the subject. Researching this article, I lost count of the number of people who demurred to go on the record, or of actors who seemed eager to speak but were then dissuaded. Fatigue might be partly to blame – it’s exhausting to be asked repeatedly about diversity because you didn’t go to Harrow and your skin isn’t white – but I got the sense that there’s more going on.

One man who passionately believes this is the screenwriter Trix Worrell, the creator of the pioneering Channel 4 sitcom Desmond’s, which brought an African-Caribbean barbershop in south-east ­London to Middle England’s living rooms in the late 1980s and early 1990s.

“TV is very difficult to break into. There’s a protectionism there,” he says with a shrug, when we meet for coffee on the seafront in Hastings, where he now lives. “People are nervous about rocking the boat.”

Though cheerful about most of the things we discuss, Worrell admits to feeling a roiling anger when it comes to this particular matter. Does he think that diversity has improved since he was pitching Desmond’s, three decades ago? “No. I say that with absolute certainty and surety.”

It is hard to underestimate the influence that Desmond’s had. The series ran for 71 episodes and at its peak it had five million viewers, remarkable for a sitcom. Starring the veteran actor Norman Beaton alongside a largely British-Guyanese cast, it made that community visible in a way that has not been rivalled in Britain in the 22 years since it came off air. It did so with the deftest of touches, addressing problems of interracial relationships and tensions within the black community through warm comedy.

“Up to that point, black people were ­never seen on TV,” Worrell recalls. “The only time we appeared in any media was in the red tops – muggings, vice. The idea was to show a black family who were just like any other.” Yet it seems that, apart from the spin-off comedy series Porkpie, occasioned by Beaton’s sudden death in 1994, Channel 4 has regarded the idea of portraying a normal black family in a sitcom as too great a gamble in the years since, despite an increase in the number of non-white roles in its other drama output.

Worrell smiles, but it is clear that the ­matter isn’t a joke. “The thing that’s said among black people is that there’ll only be one black sitcom every ten years.”

***

When I phone Paapa Essiedu while he’s on a lunch break from Hamlet, I am prepared to get a more positive perspective. Just 26, Essiedu has had a spectacular and seemingly unimpeded rise. A graduate of the prestigious Guildhall School of Music and Drama in London, he joined the RSC in 2012 and then hopped to the National Theatre in Sam Mendes’s King Lear, before returning to Stratford. The Telegraph greeted his debut as Hamlet with the notice that every actor dreams of: “A new star is born”.

But Essiedu seems ready to implode with frustration. “It’s ridiculous,” he says. “This stuff has been here for decades and decades: we’re lying to ourselves if we think there’s been a lack of awareness until now. Lots of people are talking and talking, but we need action.” Has he experienced racism directly? “Put it this way: quite often, I’ve been in a room where everyone else is white.”

A major issue, he says, is the apparently unshakeable addiction of British TV and film to corsets-and-cleavage period drama, which has left many BAME actors locked out of the audition room. The BBC is in the middle of a run of literary spin-offs, from War and Peace to The Moonstone. Over on ITV, we have had Victoria and the invincible Downton Abbey.

It still feels as though much of British drama is stuck in an airbrushed version of the country’s past. Though partly set in contemporary Egypt, BBC1’s adaptation of The Night Manager by John le Carré had only a handful of non-white actors in significant roles. Allowing for exceptions such as the BBC’s version of Andrea Levy’s Windrush-era novel Small Island, broadcast in 2009, you could be forgiven for thinking, had you never visited Britain, that people of only one skin colour live in this country. That the largely white drama series are successful on the export market only helps to extend the cycle.

“Producers say, ‘Oh, we commission stuff that people want to watch,’” Essiedu tells me. “But it’s such a narrow version of history – middle-to-upper-class Caucasian men, generally. Period drama can be from anywhere in the world: Africa, Asia. Where are those stories?”

Drama is just a sliver of broadcasting output, but other genres aren’t much better. Journalists from ethnic-minority backgrounds have made steady progress in television newsrooms – but not fast enough, Channel 4’s Krishnan Guru-Murthy has ­argued; there is a glaring absence, however, when it comes to lifestyle and entertainment TV. The recent success of the intrepid youth TV star Reggie Yates notwithstanding, it is difficult to ignore or account for the dearth of BAME presenters in documentaries and “serious” factual programming; and no major current British chat show has a permanent anchor who isn’t white.

Adil Ray’s BBC1 comedy Citizen Khan, which focuses on the escapades of the overbearing Muslim patriarch Mr Khan and his family in the Sparkhill area of Birmingham, is a rare exception. It has just returned for a fifth season. A worthy successor to Desmond’s in its tongue-in-cheek approach to potentially inflammatory issues (the 2014 Christmas special featured the birth of Mr Khan’s grandson, Mohammad, on Christmas Day) the programme also resembles its forebear in a more depressing way: it appears to be one of a kind.

When I ask Ray why he thinks this is, he selects his words carefully. “It’s not prejudice exactly,” he says, “but in the TV business, there are a lot of formulas. If you’re doing curry, get an Asian person. If it’s hip-hop, someone who’s black. If you’re doing a walk in the countryside, or drinking tea in the Cotswolds . . .” He leaves the sentence hanging.

What appears on screen is only the visible part of the problem. Actors get cast in roles only if writers write them; projects get made only if commissioners commission them. TV and film are notoriously incestuous and competitive industries. Careers are unstable. Knowing someone who knows someone is often – too often – the only way of getting work.

According to figures produced this year by Creative Skillset, many media companies fail dismally when it comes to representation. Just 24 per cent of those in senior roles in cable or satellite firms are female; 4 per cent of employees in positions in senior terrestrial broadcast are BAME; and, if the numbers are to be believed, there are no BAME people at all working on the senior production side of independent film companies. The figures aren’t entirely robust – they rely on organisations filling in forms and returning them – but if they’re anywhere near the truth they make for grim reading.

The BBC’s statistics are more encouraging (according to the latest figures, BAME people make up 13.4 per cent of staff overall and hold 9.2 per cent of leadership roles) but don’t include freelancers, an area in which it is reasonable to suppose that, without quotas to fill, representation will be worse. In September, the media regulator Ofcom put broadcasters on notice that they could face “harder-edged” regulation if they did not improve diversity.

Chi Onwurah, the MP for Newcastle upon Tyne Central, who has been vocal about these matters in parliament, says that the BBC has a special duty to up its game. “It’s not doing enough,” she tells me. “If it was, there wouldn’t be a problem. It was very interesting watching the [European Union] referendum; all the efforts broadcasters have gone to to make sure there was balance. If they went to half that effort for BAME, gender and disability, it would be a different world.”

The BBC is keen to show that it is paying attention. Last year, it appointed Tunde Ogungbesan as its new head of “diversity, inclusion and succession”, and in April his team announced eye-catching targets: gender parity across every part of the corporation; 8 per cent of staff disabled; 8 per cent of staff lesbian, gay or trans; 15 per cent of staff from BAME backgrounds. Those numbers will be replicated on screen, lead roles included, and are roughly equivalent to averages for the overall population of Britain.

Yet the idea that established BBC presenters will go quietly seems optimistic. Take the ruckus that the comedian Jon Holmes recently raised when his contract with The Now Show (Radio 4) wasn’t renewed. Holmes asked in the Mail on Sunday: “Should I, as a white man . . . be fired from my job because I am a white man?”

Ogungbesan – a former head of diversity for Shell – has a businesslike attitude to the challenges he faces, which are, he concedes, considerable. “We’ve got four years to do this, and we know there’s a hell of a lot of work to do.” That is why his team has given itself a deadline. “Hopefully, when we hit those targets in 2020, we’ll be the most diverse broadcaster in the UK.”

How does he respond to Onwurah’s suggestion that the BBC is skilled at announcing targets but less good at making change happen? “We’re publishing our results,” he says. “You’ll be able to hold us to it.”

And what if the targets aren’t met? Ogun­gbesan laughs, for perhaps a touch too long. He will not consider the possibility. “I’m like a boxer. I refuse to look at it.”

***

If British TV and film don’t get their act together soon, there may be no one left to cast. Increasingly, black and Asian stars are decamping to America to make their career there. Among those who have joined the brain drain are Archie Panjabi and Cush Jumbo (The Good Wife), David Oyelowo (Selma) and Chiwetel Ejiofor (12 Years a Slave). Idris Elba, who brooded brilliantly in BBC1’s crime procedural Luther, would likely never have been cast in a big British series if he hadn’t already made a name in the United States with The Wire. Before she appeared in Undercover, Sophie Okonedo said in an interview that the scripts she was offered from the US far outnumbered those from the UK.

Visiting Los Angeles recently, I tracked down Parminder Nagra, who made her name in Bend It Like Beckham before being spotted by a producer for the long-running medical drama ER. In 2003 she was offered the role of the Anglo-American doctor Neela Rasgotra, which she played until the series ended in 2009. A big part in the NBC crime drama The Blacklist followed, along with other film and TV work.

She never intended to move, she says, laughing ruefully, when we meet at a café in a well-to-do suburb of LA populated by movie folk. She has worked occasionally elsewhere but, 13 years on, she is still on the west coast. “The jobs I’ve got, like most actors, haven’t come about in a conventional way. It’s generally because someone is open-minded enough to look at you.”

Although she is careful to make it clear that the US is far from a utopia in terms of how it portrays race, sexuality or gender on screen – she tells a gruesome tale of a white writer who sent her his attempt at an “Asian” character – Nagra senses that things are more open in the US. “It’s a bigger pond here, because of the sheer size of the country,” she says. “There are writers of colour in the UK, but what happens is that you’ve only got one or two people at the top who are making decisions about the taste of the country . . . Those people are white.”

The landscape is certainly more open in the US. Leaving aside the allegations about Bill Cosby, NBC’s Cosby Show (1984-92) was a force for good, with its focus on a middle-class African-American family and with the numerous ethnically diverse shows it made possible: A Different World, The Fresh Prince of Bel-Air, In Living Color, Scandal (the last was commissioned by the influential black writer-producer Shonda Rhimes). Back in the early 1980s, the gentle NBC sitcom Gimme a Break! – starring Nell Carter – explored issues of racism, too.

US cable and online subscription ­services are even more courageous. Netflix’s Orange Is the New Black has an ethnically kaleidoscopic cast and plotlines that vault across almost every conceivable question of gender, sexuality, body image and politics. Where it has apparently taken the BBC until 2016 to realise that families can be both black and upper middle class, ABC in the US was years ahead: in 2014 it commissioned Black-ish, which offers a subtle portrait of an advertising executive who frets that he is losing touch with both his Obama-era kids and his inner-city origins.

Nagra nods. “There still are a lot of issues here, but if you’re an actor of colour, there is more work. All those British period dramas are really well done, but there’s a yearning there: ‘Can I please just see somebody like me on TV?’”

The reason all this matters is that TV, theatre and film have a duty to show us not merely who we are, but who we can become. In Undercover, Okonedo becomes Britain’s first black, female director of public prosecutions: this may seem unlikely, given the state of the UK’s judiciary, yet seeing it on TV helps to shift perceptions. No one would argue that Okonedo’s co-star Dennis Haysbert got Barack Obama into the White House by playing a black president of the United States in 24, but perhaps it made such a world marginally more imaginable.

The time is overdue for British TV to abandon its fetish for bodices and show us what our nation actually looks like, in all its variety – and to be more imaginative about the kind of history it presents. Colour-blind casting is mainstream in theatre. Actors of various heritages appear in Pinter or Chekhov and no one raises an eyebrow.

Anthropologists argue that race and gender are forms of performance, sets of shared codes, rather than something intrinsic to who we are. Is it so difficult to imagine a Jane Austen production with performers of black or Asian heritage? Is that any harder to believe than the thousand impossibilities we witness every day in TV drama?

I ask Essiedu if he is optimistic. Yes, he says forcefully. “I have to be. Optimism is the only way we initiate change.”

When I put the same question to Nagra, she pauses to think. “I remember being asked about this when I started ER, and I was a bit tired of the issue even then. Yet here we still are.” Her expression is wry. “So ask me in ten years’ time.”

This article first appeared in the 24 November 2016 issue of the New Statesman, Blair: out of exile