The white man's burden. While colonialism took a terrible toll on the inhabitants of India, they were not its only victims. Pankaj Mishra on the men, women and children whose lives were transformed by serving Britain abroad

The Ruling Caste: imperial lives in the Victorian Raj

David Gilmour <em>John Murray, 383pp, £25</e

Nostalgia for the Raj seems to be back. British imperialists may have helped reduce India's share of world income from 22.6 per cent in 1700 to 3.8 per cent in 1952, while making Britain the paramount world power (a feat greatly admired by Hitler, who hoped Germany would exploit the territories it conquered with similar efficiency). And the consequences of imperial haste and incompetence continue to bedevil such places as Palestine, India, Pakistan and Iraq. Lately, however, the idea that empire was not such a bad thing seems to have gained conviction.

This new moral and intellectual climate has been in the making for some time. Historians, either selling the benefits of foreign conquest or underplaying its costs (people A J P Taylor once described as "chaplains on pirate ships"), seem to be chiefly responsible. Niall Ferguson, who describes himself as a "fully paid-up member of the neo-imperialist gang", praises the British empire for having made the modern world and exhorts the United States to do likewise, making one wonder how efficient were the original makers of a world that needs to be remade so often. At the same time, Ferguson fears that pampered Americans may prove ill-equipped to form a Spartan ruling class, and give to Afghanistan and Iraq the kind of "enlightened foreign administration once provided by self-confident Englishmen in jodhpurs and pith helmets".

It is hard not to suspect that the neo-imperialist gang is actually a Kipling book club, longing for heroism and grandeur in a world increasingly contemptuous of western power. It was Kipling who created the romantic imperialist legend of the British soldiers, administrators and engineers in the colonies as carrying heroically the white man's burden - the legend that neo-Raj fabulists at present retail, feeding on general ignorance and a liberal guilt about empire that is as shallow as it is vague.

Happily, David Gilmour and Vyvyen Brendon, authors of two new books about the Raj, stay clear of facile arguments about whether the British empire was good or bad. The biographer of Lord Curzon and Kipling, Gilmour seems less interested in either regretting or celebrating the past than in exploring how civil servants in the Victorian Raj saw themselves. He may not disagree much with the line on imperialism that George Orwell took after his experience of Burma under British rule:

The Burmese are under the protection of a despotism which defends them for its own ends, but which would abandon them without hesitation if they ceased to be of use. Their relationship with the British Empire is that of slave and master. Is the master good or bad? That is not the question; let us simply say that this control is despotic and, to put it plainly, self-interested.

But, as Gilmour puts it, "conceding the coercion and the self-interest" behind their presence in India "did not dent the Victorians' belief in the righteousness of their rule". They continued to parade "their roads and railways and canals, their system of justice, the medical and sanitary improvements".

Gilmour describes this ambiguous achievement in full and satisfying detail. Almost everything is here, in what will replace Philip Mason's The Men Who Ruled India as the definitive book about the British ruling class in India: the class gradations within the Indian civil service; its ideology, doers, slackers, scholars, thinkers and eccentrics; people coping with the summer exodus to the hills; the intransigent tribes on the border with Afghanistan; dissolute Indian princes.

Gilmour occasionally admires the Victorian civil servants for their sense of moral responsibility - what Orwell thought that an Anglo-Indian such as Kipling felt more acutely than such liberal English visitors as E M Forster, and what often made the most destitute of Indians see their British rulers as their mai-baap (literally "mother-father"). He persuasively defends the Victorian scholars against Edward Said's allegation that the fact of British power compromised "all academic knowledge about India" (Said's remarks on India are not among the most convincing parts of Orientalism). Gilmour also treats sympathetically the much-scorned memsahib. As he puts it, for women living in remote areas, "the environment did not encourage a broadening of minds or a generosity of spirits".

At the same time, Gilmour is critical of inept and unfeeling administrators, especially those who let their minds be addled by imperial or free-market ideology. He describes how, in 1865, British officials shot down a plan to import rice to deal with likely famine in the eastern province of Orissa because they saw it as interfering with the laws of supply and demand. Partly as a result, nearly a million people died of starvation and disease.

Many such famines, small and big, occurred in India largely because of the British obsession with growing export-oriented commercial crops. Present-day Indian nationalists may also have to admit, however, that India's current neoliberal rulers hold up only slightly better under scrutiny: hundreds of people still die of starvation in India, despite the country's self-sufficiency in food; and, in a tragedy without precedent in the history of British India, failed crops, bad debts and an indifferent government have resulted in recent years in thousands of suicides among farmers.

The most moving part of Gilmour's book deals with civil servants in suburban retirement in Britain. Travelling far from often modest origins, many of these men had ruled territories bigger than some European countries. Yet their compatriots in Britain took little interest in the empire, or the men who ran it on their behalf. Living in what seemed to many an alien land, the retired civil servants missed "the sound of frogs, the flame of the forest, the almond groves of Kashmir". They often tried to "compensate by setting up little Anglo-Indias in their houses". They worried about being "bores" and, as Gilmour writes, mostly "went downhill in obscurity, doing the Times crossword and muttering about the modern world".

This desolation may have been part of the price they paid for holding on to their Britishness in India - a Britishness that was past its sell-by date in the Britain to which they returned. Settling down in India, or embracing Indian ways, was never really an option for the British in India who, as Gilmour writes, "wished to remain British". In fact, much like Indian and Pakistani minorities today in Britain, they feared the loss of their cultural identity: they worried that their children, as Vyvyen Brendon writes, "might, like the children of Dutch, French and Portuguese colonials who stayed in India with their parents, become native in their minds, if not in colour".

Brendon has carved a rich narrative out of memoirs, journals and oral testimonies. Her subjects include the full range of British people in India - the civil servants as well as those in less lofty positions. Though often dense, her book is full of fascinating information about the methods and institutions the British improvised in order to cope with life in India - how, for instance, "India led the way in providing cheap elementary education for deprived children". But Children of the Raj is at its strongest when evoking the emotional history of children born in the Raj: how the British, who at their best were mai-baap to millions of Indians, became absentee parents to their own children.

Brendon names the most famous children expelled from idyllic Indian childhoods into the harsher world of striving and achievement: Cliff Richard, Julie Christie, Lindsay Anderson, Vivien Leigh, Spike Milligan, Tom Stoppard. She also describes the experiences of innumerable others, the young-est of whom are now in their sixties. In 1996, when I first travelled to England, I was always surprised to meet people with colonial backgrounds and connections, such as Alan Ross, editor of the London Magazine, for whom India remained "a yearning that would not go away". Like Ross, many British people enjoyed idyllic childhoods in India, their formative years spent among fabulous sights and caring ayahs. Kipling, born and brought up in Bombay, wrote in his memoir Something of Myself: "Give me the first six years of a child's life and you can have the rest." Anxieties about class and status however forced many imperial servants to send their children to England, where they were fostered or sent to boarding schools.

This uprooting generally bewildered the children. Kipling's short story "Baa Baa, Black Sheep" evokes powerfully the five years of exile that he spent with extremely cruel guardians in England, suffering partial blindness and what seems to have been a nervous breakdown. As Kipling's sister, Alice, wrote: "I think the real tragedy of our early days . . . sprang from our inability to understand why our parents had deserted us . . . it was like a double death, or rather, like an avalanche that had swept away everything happy and familiar." The experience certainly damaged Kipling, who never really felt at home in England. Equally unmoored from India, he chose to base his unsettled identity on the certainties of a pious imperialism. Although others dealt more rationally with their sense of alienation, their inner lives could not but be shaped by the Raj. If the British changed India, they were also changed by it, linked to their subjects by unconscious bonds of suffering, which prove - if proof were needed - that the making of the modern world leaves no one unscathed.

Pankaj Mishra's most recent book is An End to Suffering: the Buddha in the world (Picador)

This article first appeared in the 19 September 2005 issue of the New Statesman, The gathering storm