Goebbels kept a diary, but (as the Sunday Times found to its cost) Hitler didn't. Queen Victoria painstakingly wrote hers, but it is a safe bet that her great-great-grand-daughter, our present monarch, doesn't. One can't imagine Mick Jagger bothering. (He, one hopes, follows the school of Tallulah Bankhead: "Only good girls keep diaries. Bad girls don't have time.") Andy Warhol, however, used to spend his mornings on the telephone to a special diary amanuensis, outlining his previous day's activities. (He subscribed to the alternative, Oscar Wilde principle, enunciated in The Importance of Being Earnest: "I never travel without my diary. One should always have something sensational to read in the train.")
Through various governments, Richard Crossman, Tony Benn, Alan Clark and Paddy Ashdown have noted the triumphs and posturings of their parliamentary colleagues. Douglas Hurd kept diaries (as yet unpublished) during the last Conservative administration. But who is doing his or her bit for posterity today? Mo Mowlam and Alastair Campbell have been mentioned as possible candidates, and publishers salivate at the thought of Peter Mandelson's jottings. Yet nobody can admit to such an antisocial act.
Politicians can at least be sure they will gain financially from their efforts, but that does not necessarily explain their motives. "I sometimes wonder why I keep a diary at all," pondered the Tory MP Sir Henry "Chips" Channon in 1936. "Is it to relieve my feelings? Console my old age? Or to dazzle my descendants?" His near contemporary Virginia Woolf said it more succinctly: "I wonder why I do it."
Most of us sympathise, our diurnal scribblings never having progressed beyond a record of the first few days of successive Januarys in the years of our middle adolescence. But others get the habit which, Harold Nicolson confessed to his son Nigel, is "like brushing your teeth".
Diaries are written partly to satisfy private emotional needs, and partly with an eye to the future. For historical reasons, women have tended to the former mode (diaries offering an outlet where self-expression is stifled) and men to the latter. Showing typical maturity, the Dutch teenager Anne Frank spoke for many of her sex when she began her journal in June 1942. Unhappy at the triteness of her contemporaries in wartime Amsterdam, she recalled the adage "Paper is more patient than man" and commented: "I want to write, but more than that, I want to bring out all kinds of things that lie buried deep in my heart."
Half a century later, a very different kind of diarist, Alan Clark, described the same process from a male perspective as "a baring, if not a flaunting, of the ego". He was closer to the mark than he imagined. A good diary allows one to play with ideas, adopt different perspectives, put forward views one would normally self-censor. In the process, one can arrive at an alternative, and perhaps more truthful, sense of one's being.
A diary also allows one to reclaim the past. It acts as an aide-memoire, summoning a forgotten name, colour or mood. Thus, in March 1768, Fanny Burney resolved to address her ideas to a certain Miss Nobody (that is, her diary): "To have some account of my thoughts, manners, acquaintances and actions, when the hour arrives in which time is more nimble than memory, is the reason which induces me to keep a journal."
And if one wants to remember things for oneself, why not for eternity? As Burney also noted, after two diary-less months, "But for my pen, all the adventures of this noble family might sink into oblivion!" Public figures, whether artists or politicians, would agree. But if they try too obviously to give their own version of history, the result falls flat.
In A Book of One's Own, the best overall study of the genre, the American writer Thomas Mallon suggests three starting points for the modern diary. One is the late medieval commonplace book, in which members of a small literate clique would post their favourite religious texts. Gradually, these bookish scavengings became more secular. Milton's digest of his reading anticipated the personal notebooks kept by creative types such as W H Auden, who called his A Certain World "a map of [his] planet".
Another source is the journals kept by Protestant divines, from the 16th century, and used as a spiritual aid for examining their consciences. At times of persecution, their jottings were swapped among themselves, partly as an act of solidarity, and partly as a form of art, thus taking on the subversive nature of a modern samizdat.
A parallel development was the daily records kept by Elizabethan explorers as they mapped out the world. Their diaries were practical tools, accurately describing the features of places they visited and so assisting anyone who came after them.
All three strands remain essential - the examination of ideas (as in Edmund Wilson's journals), the confessional (found in spiritual reflections such as C S Lewis's A Grief Observed) and the objective account of external reality (anything from Robert Byron's travel diaries to Hermione, Countess of Ranfurly's wartime escapades).
With his first entry in January 1660, Samuel Pepys invented the modern diary. He pioneered two features of the genre. One is the need for a strong, individual voice whose diary reveals exactly what he thinks, how he conducts his relationships and what he sees around him. He is his own man, saying and doing as he pleases.
This was the approach of Alan Clark who, while admitting his egotism, defended himself (in his first volume of diaries): "Sometimes lacking in charity; often trivial; occasionally lewd; cloyingly sentimental, repetitious, whingeing and imperfectly formed. For some readers the entries may seem to be all of these things. But they are real diaries."
Not everyone flourishes at the centre of attention. A rampant ego may create a unique world, but can obscure understanding of others. So the flip side of the Pepys and Clark coin is the quiet, dispassionate observer, epitomised by the gentlemanly James Lees-Milne or the eagle-eyed Frances Partridge, whose well-crafted journals are valuable resources on manners and morals. Other modest but equally telling chroniclers include the Germans Count Harry Kessler and Victor Klemperer.
Pepys's second achievement was establishing an ideal perspective for the diarist. He should enjoy a privileged but untaxing position in power, although at a slight tangent. He should be friends with ministers and literary honchos, but not one of them. He can then master two central diary illusions: offering a sense of familiarity with the inner workings of a group, while maintaining a cynical - or at least detached - overview. Pepys managed this through his role as a senior Admiralty official. His mantle was taken up by John Evelyn, sometime commissioner of the privy seal, and, in France, by the dis-affected courtier Louis de Rouvroy, Duc de Saint-Simon.
In the early 19th century, the best observer of literary- political fashion was Charles Greville, a clerk to the Privy Council. Dining with Lord Holland in February 1832, Greville met the historian Thomas (later Lord) Macaulay for the first time. He was soon disillusioned: "It was not till Macaulay stood up that I was aware of all the vulgarity and ungainliness of his appearance; not a ray of intellect beams from his countenance; a lump of more ordinary clay never enclosed a powerful mind and lively imagination." No wonder Lord John Manners likened Greville to "Judas writing the lives of the Apostles".
The list of such privileged insider-outsiders contains leading 20th-century diarists: the likes of Sir John Colville (Winston Churchill's private secretary, who even called his journals The Fringes of Power), Harold Nicolson (the diplomat and MP), Chips Channon (the rich, bored backbencher), James Lees-Milne (whose job as adviser to the National Trust provided a unique observation post on the British landed classes) or Gyles Brandreth (with his wrily comic view from the whips' office).
Such diarists self-reverently regard themselves as part of a tradition. Alan Clark read Channon for pleasure each morning (significantly, he described his predecessor as "snobbish, arriviste but intensely observant"); Channon felt haunted by the spirit of Horace Walpole.
But for every successful diarist, many more go unrecognised. Paradoxically, at a time when people no longer have time for reflective communication such as letter writing, diaries are hugely popular. Schoolchildren are required to write journals of their holidays; therapy patients of their dreams. Publishers latch on to anything, from a cricketer's "diary of the season" to Ossie Clark's meanderings. Their enthusiasm is not misplaced. The public enjoys diaries as a form of spectator sport, where they can watch James Boswell toying with a poxy whore in Georgian London, or Joe Orton lifting the lid on theatreland. If a record is unavailable, it is simply made up, as in the fictional exploits of Adrian Mole or Bridget Jones.
Technology now threatens this literary effort. When Big Brother participants wanted to let off steam, they went into a wired "diary room". (Imagine such rooms dotted around Britain's high streets like photo booths.) And who needs a written record when people stick digital cameras in their bedrooms and broadcast the results on the web?
One answer is biographers and historians, who, occasionally, are rewarded with a complete evocation of a literary culture, as found in Virginia Woolf, or with a window on to one strand of society - the legacy of Evelyn Waugh. More often, they scour diaries for titbits of information on lesser-known personalities (the stuff of Woodrow Wyatt's entertaining, if self-serving, oeuvre); for period detail (as found, for example, in Roy Strong's skilful juxtaposition of bluff Michael Foot and urbane Lord Goodman at a 1973 arts dinner); and for nuance (such as Noel Coward's reticence about sexuality in his otherwise barbed journals).
Diaries have been used to effect in recent lives of Fanny Burney, Mary Shelley and even Sir Mansfield Cumming, the first head of MI6. But they are not essential: Oscar Wilde left none for Richard Ellmann. In my biography of Rudyard Kipling, I made do with a bowdlerised version of his wife Carrie's diaries. She left instructions, on her death, that they should be burnt. But an earlier biographer, Charles Carrington, had been permitted to view them and take notes. So his clipped version of her diaries now provides a curious, flickering and, ultimately, rather appropriate record of the Kiplings' married life.
Carrie's pyromania suggests exactly why diaries are so attractive. They are, as the French say, journaux intimes, or receptacles for secrets. Chips Channon admitted in July 1935: "I feel caddish, even treacherous sometimes, keeping this diary from the eyes of my wife - yet it is our only secret. She knows I keep it, but if she were to read it, and I knew she were, it would lose much spontaneity, and cease to be a record of my private thoughts . . ." Three days later, he wrote: "I am bored by this Italian-Abyssinian dispute, and really I fail to see why we should interfere . . . Why should England fight Italy over Abyssinia, when most of our far-flung Empire has been won by Conquest?" Such deft mixing of private and public arenas is irresistible.
That sense of the sanctity of diaries can also make them truly powerful. When Winston Smith rebelled against Big Brother in George Orwell's novel 1984, he wrote his private diary. (Sinisterly, the Big Brother television series got that wrong.) Diaries allow expression where it is otherwise not permitted: thus the thriving literature of diaries from totalitarian societies and from prisons; thus the demand for marginalia from Cabinet ministers' notepads. Diaries even provide space for libertarian conservatives such as Alan Clark to give vent to their frustrations. As with medieval divines, the best diaries are by iconoclasts and subversives.
Andrew Lycett's Rudyard Kipling is published in paperback by Phoenix Press (£9.99)