The alchemy that drives literary fashion is as mysterious and inexplicable as that of haute couture. More so, indeed, for you can predict with fair accuracy that if last season's summer collections had a nautical theme, this season will be all about ruffled florals.
Yet who could predict or explain the tectonic shifts of literature that produced the vast, multi-volume biographies of the late 1960s and 1970s (such as Michael Holroyd's works on Augustus John and George Bernard Shaw); the nest of fictional singing birds that contained Ian McEwan, Kazuo Ishiguro, Martin Amis and Julian Barnes; or the Bridget Jones tsunami that continues to flood bookshops with chirpily introspective women's fiction? Only a few years ago, if you'd been asked to name a truly unfashionable genre, your answer might have been memoir. (Next to epic poetry - and hang on, wasn't Clive James writing some of that? Which presumably made it, in some way, quite modish?)
In 1825, when the courtesan Harriette Wilson published her pioneering kiss-and-tell memoirs with their explosive first line (still one of the great literary come-ons of all time) - "I shall not say why and how I became, at the age of 15, the mistress of the Earl of Craven" - memoir may have been a genre with the power to intrigue.
But over the next 150 years, the dead hand of pointless reminiscence squeezed much of the spirit out of the genre. With a few remarkable exceptions (such as J R Ackerley's fiercely truthful writing about his life), candour and revelation fled. Memoir was annexed by rambling military men and anecdotal aristocratic ladies, to the point where the author Mary Dunn had a riotous success in Punch magazine in the 1930s with her parody Lady Addle Remembers:
I remember once seeing Mipsie lying in a dahabeeyah in broad daylight, dressed only in purple sequins, with a goat bending over her. I've no idea what she was doing . . .
Half a century later, when Tobias Wolff began writing This Boy's Life (1989), little had changed. "The memoir field was pretty much commanded by eminent actors and military men," Wolff said in a recent interview. "There was no reason for anyone to be interested in me because of who I was. I had a respectable readership as a short-story writer. I was doing quite well. But I had no idea this book would take off the way it did."
Wolff's breakthrough was to find a way of writing about himself that combined the authentic voice of truth with the artistic rigour of fiction. Or not find, so much as rediscover the impulse for a writer to explore large truths about humanity through the close study of the minutiae of his own life, of which Montaigne's Essays (1580) are still the most brilliant example. "My concern," Montaigne wrote, "is not to depict the individual as he exists, but to show him in the act of becoming. I paint the passing of time - not from one age to another, in seven-year stages, as people say - but from day to day and minute to minute. My story changes with the passing hour."
That bold candour was, for a very long time, the preserve of fiction. The novel evolved as a way of exploring obliquely truths about society and individual sensibility too dangerous and unpalatable to confront directly.
A favourite device of novelists, from Swift and Defoe to those of our own day, has been to frame their fiction in the guise of memoir. But it doesn't necessarily work the other way round, as James Frey discovered on the publication of A Million Little Pieces, his purported memoir of addiction, which turned out to contain what his erstwhile champion Oprah Winfrey considered to be more than the permitted amount of fictional additives. Not that it did him much harm in the end. Fewer than a couple of thousand readers took up his publisher's offer to return their money, on receipt of an affidavit stating that they had bought his book under the impression that it was true.
This leaves the nature of the relationship between fiction and memoir as enigmatic as ever. That an intimate connection between the two exists is certain: among the remarkable memoirs to be published in the past few years, many of the most striking are by novelists - Hilary Mantel, Barnes, Amis, Elizabeth Jane Howard, Luke Jennings, Candia McWilliam and Julie Myerson.
Even the reticent Annie Proulx has been tempted to reminisce lately. Her memoir, Bird Cloud, her first work of non-fiction in 20 years, is an account of her purchase of 640 acres of wilderness in Wyoming, on which she built a house that she hoped would be both a work of art and the place where she might settle, perhaps for ever.
In the end, the project failed - as does the book; for Proulx, perhaps ill at ease with allowing herself into her own narrative, retreats into something that reads like a tetchy combination of a local guidebook and an extended lament about unsatisfactory builders:
Harry's roof engineer, J Horne, decided that someone at the truss builders had miscalculated the load where the beams tied together. So now J Horne and Jim were rethinking which beams must be used. Jim told Gerald that he would let him know the next day whether we could order and wait for the beams, or whether the beams were in Denver, or whether we had to have the beams manufactured. For once Gerald was upset . . .
And so, interminably, on. The trouble with writing memoir was pithily summarised by Montaigne. "Elsewhere you can commend or condemn a work independently of its author," he wrote, "but not here: touch one and you touch the other."
It takes, as I know from my own experience of writing a memoir of middle age, a curious act of double-think for an author to parade about naked in this alarming fashion. On the one hand you write, as best you can, the truth. But while working on my book I found that I could only do it by engaging in a writer's version of the childish fantasy that no one can see you if you shut your eyes.
Critical reactions to memoir are, almost necessarily, ad hominem. However hard a critic tries to review the book (and many of them don't try very hard at all), he or she finds it hard to avoid assessing the personality of the author. At worst, this can lead to the kind of pack-mentality spite that surrounded the publication of Myerson's The Lost Child (2009), about her son's drug addiction - though even that raised interesting questions about the ownership of narrative.
The troublesome - and exciting - thing about memoir is that there are no rules. It is easy to be daring in a novel because there are no consequences. If Myerson had written her memoir as fiction, there would have been no row: just a fistful of respectful reviews praising its sensitivity and elegance. Even now, there persists a school of literary thought that believes in the novel as the "ideal" form.
Among the reviews of my own memoir, I was startled by several which suggested that since I wrote quite nicely, it was time I got on with something a bit classier, in the form of fiction. "Would make a great novelist," said one. "[Her] thoughts on love and ageing . . . would grace a literary novel," said another. "If this woman wrote a novel, I'd buy it," said a third.
Inevitably, there is something gladiatorial about the public appetite for "truth". Literary memoir exists in a continuum, not just with celebrity memoir, but with such ingenious blends of artifice and "reality" as I'm a Celebrity . . . and Big Brother. The impulse that drives us to read gossip columns, tabloid newspapers or Private Eye is exactly the same as the one that leads us to pick up books by McWilliam, Joan Didion or Tony Judt. Curiosity is its name. We want to know what it feels like to suffer loss: to be grievously bereaved, blind and becalmed in a hospital bed, unable to move but with our beautiful mind intact.
It is a primitive instinct - vulgar, even, you might say - but it is also the stuff of survival. Curiosity was the raw material of every discovery human beings have ever made. On a literary level, it teaches us that most basic and essential of human lessons: how to live.
“The Stranger in the Mirror" by Jane Shilling is published by Chatto & Windus (£16.99)