Scandal. The newspapers full of stories about child abuse going unchecked. Campaigners, determined to shine light into areas that had hitherto remained murky.

Not November 2012, but October 1888. John Tobin, “a hard-working man” who “generally got drunk on Saturdays”, had been brought to a magistrates’ court in the East End of London, accused of leaving his two-year-old son, Daniel, filthy and starving in a pitifully cold family home. Neighbours testified that they had often been forced to throw food through a window to Daniel and his four siblings.

But John Tobin and his wife, who was also accused, walked free, to the fury of the nascent National Society for the Prevention of Cruelty to Children (NSPCC). The prisoners, the magistrate said, had no doubt neglected their children, but he could see no way to convict them. A clause under the poor laws did say that parents should nurture their children, but it was designed entirely to protect the public purse rather than the children. So long as the Tobins did not present their offspring to the Board of Guardians for poor relief, their family life would remain sacrosanct.

No sooner had the guilty couple walked free from the court than Rev Benjamin Waugh, a co-founder of the NSPCC, was dashing off a letter of protest to the Times. “Had it been John Tobin’s dog which was in question,” he wrote, “no difficulty would have arisen, for the law is clear as to starving dogs.”

The recent history of childhood can be seen, if you like, as an ongoing battle between two opposing ideological camps. On the one side have been ranged those who subscribe to a sort of post-Renaissance notion of the child: that he or she is a vulnerable innocent, in need of ever greater protection. On the other side have been those who lean towards the pre-Renaissance view that children are somehow flawed – even evil – and in need of correction. And while the pendulum of public opinion has continued to swing between these two extremes over the past 120 years, most of the legislative travel has been in one direction.

Waugh and his fellow campaigners – believers in the post-Renaissance child, if ever anyone was – were able to use the Tobin case, and others like it, to good effect: 1889 marked the passing of a law known as the Children’s Charter, which enshrined clearly in law for the first time the right of the state to pass through the front door of the family home and intervene in relationships between parents and their children. The act started a process that continues to this day, with every national scandal or perceived failure to protect children addressed through additional legal protection. The aim of every reform has been the same – to improve children’s chances of growing up strong in every respect. And yet the effect has been broader than that. In many ways, the years since 1889 could be seen as the period when the state gradually took over much of the role hitherto played by the family in children’s lives. The 20th century was the century during which the state stepped in; when gradually its influence and its powers crept in through the windows and under the doors of the family home.

The mythology surrounding childhood has very often been driven by fear, particularly over the past century or so. In the pre-Renaissance camp, there has been concern that a failure to correct children’s innate flaws has allowed illiteracy, or even violence, to flourish. On the post-Renaissance side, the fears have been mostly about external threats such as abuse and poverty. And as the state, rather than the family, has become the chief protector of children, a new fear has crept in, perhaps on both sides – are our children’s lives now too constrained, too cosseted, too lacking in independence, as a result of all this extra interference?

In the mid-1880s, such fears were not a big concern, and for the most part family life remained sacrosanct. Even child sexual abuse was barely the concern of the courts. Incest was known about and widely condemned, the perpetrator often becoming a “marked man” in his neighbourhood. Until 1889, however, wives were prevented from giving evidence against their husbands, and so it was exceedingly hard to prosecute this crime. In one case, in Lambeth in 1880, a woman even brought her friends and neighbours into her house so that they could witness her husband abusing their seven-year- old daughter.

On paper at least, the 19th century had been a time of increasing state intervention in children’s lives. Their education had become compulsory, their working lives increasingly regulated and their health a matter of growing concern to a nation preoccupied by the need to maintain control of its empire, which it could not do if so many of its young died before they reached maturity. And yet as the Victorian era slid into the Edwardian, it was charities, rather than the government or local authorities, which continued to make children’s lives their day-to-day concern.

Such a plethora of benevolent organisations had sprung up in the cities that there were letters to the press fretting that parents were being allowed to hand over their responsibilities: the Ladies’ Sanitary Association, the Infant Health Societies of Marylebone and St Pancras, the Police-Aided Association for Clothing the Destitute Children of Liverpool, the Children’s Country Holidays Fund . . . The list went on.

“Instances occur of the same children being sent away twice, or even thrice, in one summer, by the different agencies,” a correspondent fretted in the Times. In 1896, the Charity Organisation Society held a special meeting to consider the problem of over-charitisation in central London. “There was very little real poverty [in the district] except that which was the result of loose and shiftless habits,” one participant explained.

Despite the legal advances in the 19th century, it was well into the 20th before the state really began to take its responsibilities to heart. A turning point came just three weeks after VE Day in May 1945, with the publication of the Monckton report into the death of a boy named Dennis O’Neill.

In June 1944, Dennis and his brother Terry, whose parents had been convicted of neglect, were placed with a couple, Reginald and Esther Gough, at their farm near Minsterley in Shropshire. In January the following year, Dennis died of heart failure after being struck repeatedly in the chest. His back bore the scars of beatings with a stick, he was painfully thin and he had septic ulcers on his feet. Both Goughs were charged with manslaughter. Terry told the court that he and his brother had lived mainly on bread and butter, along with any milk they could suck from the cows’ teats.

The death of Dennis O’Neill, or rather the general reaction to it, underlined a change in public attitudes to children and family life, and signalled the way to the next half-century of public policy. What set this death apart from the countless other deaths of neglected and abused children was that the central failure, in the eyes of the world, was not that of the individuals involved – though that had indeed been grievous – but of the state.

By the time the Monckton inquiry produced its report most of the building blocks of the new welfare state were in place and huge policy changes were under way. But the case helped to validate the feeling that, in future, the government would play a much greater part in the nation’s life – and its involvement would not stop, as it had continued largely to do up until now, on the front doorstep of the family home. A century of social and legislative change was beginning, finally, to permeate the public consciousness.

In a specific sense, it gave rise to the Children Act 1948, which gave local councils the right to assume parental responsibility over a child, and led to the burgeoning of social services departments. In 1952, the state gained the right to remove children from the family home even if their parents had not been prosecuted for cruelty; and in 1958, judges gained the power to order that children be taken into care in divorce disputes. Later, the tide began to turn and more emphasis was placed on preventive work to help families stay together – but in some respects the broader effect was the same: when family life went awry, it was increasingly the state’s job to step in and sort out the mess.

But as a young woman named Pauline Colwell gave birth in March 1965 to a daughter named Maria, there was still a feeling that the system was not putting the needs of individual children first. Certainly the Sussex social workers who handed little Maria back to her mother in Brighton in October 1971 after six years in foster care were not acting in accordance with her wishes. Until then, Maria had been looked after by her uncle and aunt Bob and Doris Cooper. And when her mother’s demands for her return became insurmountable, Maria could be persuaded to go only by telling her a lie – that she would be allowed to visit the Coopers. She never saw them again.

In the last year of her life Maria was used by her mother and stepfather, William Kepple, as a drudge. Neighbours would see her dragging heavy bags of coal back home from the shop and would hear her mother slapping her and calling her a “dirty little bitch”. Yet there was a feeling that the authorities knew best. Maria’s teacher said later that, although she had felt the manner of the girl’s return to her mother was inhumane, she assumed it was for the best. On 7 January 1973, the Kepples arrived at the local hospital with Maria’s limp body in a pram. She weighed just 36lb – 10lb less than the normal weight for a child of her age. William Kepple was convicted of manslaughter.

Neighbours and other local people – some of whom had tried repeatedly to raise the alarm – queued to get into the public inquiry that was subsequently held. The inquiry was swingeing in its criticisms of social workers but went even further: “It is upon society as a whole that the ultimate blame must rest; indeed the highly emotional and angry reaction of the public in this case may indicate society’s troubled conscience,” it said. “It is not enough for the State as representing society to assume responsibility for those such as Maria.”

When Dennis O’Neill died at the hands of his foster parents, social workers were said to have failed him. When Maria Colwell died at the hands of her stepfather nearly 20 years later, society as a whole began to question how it treated its young.

The protection of children has always been driven by scandal and fear, and this was particularly true in the 1970s. Indeed, the 1970s were demonstrably a dangerous time for children. Perhaps it was because a newly liberal atmosphere around children was being encouraged by the growing notion that young people were individuals, with “rights”. Perhaps it was because somehow, as the state had stepped in, parents had stepped back. But there were subcultures in which sexual abuse was able to flourish: around the BBC, we now know, and in children’s homes across the land.

There was grave risk to children’s lives, too. The 1970s were a more dangerous decade for children, in terms of vulnerability to crime, than any other before or since. In 1974 the child murder rate rose to an all-time high of 200 a year – almost four times the level at which it stands today.

Another succession of abuse cases and violent deaths shocked the nation: 21-month-old Tyra Henry, battered and bitten by her father while in local authority care in 1984; three-year- old Heidi Koseda and four-year-old Jasmine Beckford, both beaten and starved by their stepfathers in the same year; four-year-old Kimberley Carlile, killed by her stepfather in 1986; 16-month-old Doreen Mason, who died in similar circumstances in Southwark in 1987. And so on.

Yet with each renewed outbreak of public revulsion and outrage, children as a corporate body would garner a little strength. As children became less the appendages of their parents; they increasingly became individuals. It would take another 20 years or so before children would have those individual rights enshrined in law, but the process had begun.

In other arenas, too, the powers of the family were being diminished. One woman came to personify that change: Victoria Gillick. A Roman Catholic and a mother-of-ten, Mrs Gillick demanded an assurance of her local health authority in Cambridgeshire that it would not offer contraception to any of her daughters, all of whom were under the age of 16, without first asking her permission. When the health authority refused to give any such assurance, she took her complaint to court. The case went all the way to the House of Lords. And, in 1985, Mrs Gillick lost.

The case had far-reaching implications. Mrs Gillick was a campaigner for the rights of the parent; a campaigner for the notion that a child was essentially the property of its mother and father. But the House of Lords ruled that parents did not have the right to a say over their daughters’ lives. The debate began to spin outwards – where should it all stop? Was it practicable, or even desirable, for the state to prosecute parents who allowed their children to drink, smoke, read porn or watch violent films?

What were adults’ responsibilities towards children? To protect and nurture them, or to set them free to make their own mistakes? To shelter them from the harsh realities of the adult world, or to initiate them so that they might be stronger? As the 1990s began, these questions hung heavy in the air.

One event, three years later, would begin to provide the answers. Extreme occurrences in which the entire nation joins in shared and powerful emotion are bound to shake out something fundamental about the way in which a society is operating, and the death of James Bulger in February 1993 was no exception. In an age characterised by adults’ guilt over their conflicted emotions about children, about child-rearing and about the place of the child in the wider world, the question that was asked was not really a question about what Robert Thompson and Jon Venables, who were both aged ten, had done. It was: what have we done? It was a question asked loudly, repeatedly and with great anguish. And it marked the sudden realisation of this notion which had been growing under society’s surface for years – that children were the responsibility not merely of their parents, nor of the state, but of the whole society.

A few days after James Bulger’s murder William Golding, the author of the classic 1950s novel about child savagery Lord of the Flies, gave voice in the San Francisco Examiner to a fear lurking in many hearts – that somehow we were all to blame. “Where the orders and patterns of society cease to matter, gangs begin to find cohesion merely in the joint fulfilment of their darkest instincts . . . If parents are absent, if fathers do not provide strength and mothers do not provide love, then children will plumb the depths of their nature.” Even before the perpetrators of the murder had been identified, the verdict had been passed: the fabric of civilisation was perishing, and children – all children – were the victims.

With that sense of collective responsibility came a feeling that there was a need for grand, collective action. The advent of New Labour shifted responsibility for making that happen once again on to the state –where the state itself did not take it up with gusto. As Britain ratified the UN Convention on the Rights of the Child in 1991, children gained various grandiose “rights” – to have special protection, to be able to pursue their talents, to participate in achieving a better future for all children. There was almost nothing, it seemed, that the state could not now promise the child. The British government, underlining an article of the UN convention, even promised its children the right to “grow up in an environment of happiness, love and understanding”, though how this was to be achieved was never made clear.

A wide range of more practical schemes was conceived, too. By 2008 the targets towards which the government’s department for children and schools would be working included breastfeeding, childhood obesity, bullying, social care assessment, preventable child deaths, exam performance, drug misuse, teenage pregnancy and youth crime, to name just a few. The government set a plethora of targets, and missed. And yet, with every one, the sense of impending crisis that surrounds the nation’s children today continued to grow.

Never before have governments had so much to say and so much to do about children. The child has become a focus for public policy in an unprecedented manner. Yet at the same time this chorus of fear and concern around childhood has crescendoed. It is as if both narratives in the great childhood debate – pre-Renaissance fears about moral and personal decline; post- Renaissance worries about welfare –have united to form one voice. And that voice is saying that childhood is in crisis.

Children, when dragged into the amphitheatre of public debate, have always been an emotionally charged subject. But in recent years the adult world has almost seemed to need those fears. In the past 20 years they have popped up in every sphere – education, health, leisure, diet. We worry about the pressures on children: pressure to achieve at school, pressure to be slim, pressure to have the right stuff, pressure to have seen the right films; to be able to achieve the right levels in the right computer games. We worry, just as the Victorians did, that children are somehow going to the dogs.

That fear, examined closely, derives from a central concern: if children are the future, what sort of future will they be? The endless fretting – about abuse, about unhealthy lifestyles, educational underachievement, criminality – is in fact fuelled by fear of the unknown. Everyone assumes that his own family is doing a good job . . . but is someone else sneaking rotten apples into the barrel? Will the next generation be strong enough to protect and nurture this one in its dotage?

Perhaps the great mystery about children today is not why they continue to suffer from abuse, neglect and overindulgence. It is why the adult world continues to worry so intensely about these things. After all, there is much to celebrate. A child born in the UK in 2010 can expect to live to be 80 years old; the average child born in 1910 lived to the age of 50. Of every 1,000 children born in the UK in 2010, just five will die before their fifth birthday; a hundred years earlier, 140 of every 1,000 did not survive infancy. Before the Second World War, three in every 100 children went to university; by 2010, about one in every two. The reported crime rate began dropping from the mid-1990s. Suicide and self-harm are becoming less common. Even unemployment is nowhere near as high as it was in the 1930s. Children nowadays are less likely to be in lone-parent households than they were during Victorian times – but now family break-up is usually caused by divorce rather than death.

But the fears and the myths are not just fears and myths about children alone. They are based on ideas and anxieties about mankind, anxieties that have emerged repeatedly throughout the ages because they have drawn people together as external threats tend to do. In uncertain times – and times often have been uncertain – there is a reassurance to be found in worrying about abstract, unknowable dangers. If the evil, the poison, the violence, is out there, prowling around the dark boundaries of the camp, then those huddled inside nearest to the fire may take comfort in the rhythms of their own lives.

Perhaps at some point the realisation will come: we have indeed moved forward. And perhaps with that greater cohesion and clearer sense of common purpose will come the realisation, too, that, for the most part, there is little out there to fear but fear itself.

Fran Abrams is the author of “Songs of Innocence: the Story of British Childhood” (Atlantic Books, £20)