I remember telling a friend, at the start of 2006, that there was little hope in my heart that idiosyncratic, outspoken young women could succeed in mainstream music. To be big these days, I said, you have to be bland. If you make pop, you have to be bleached and plucked, writhe around on stage in something sparkly and thigh-skimming, and let your voice be digitised into a robotic whine. If you make soul, you have to embody sex of a rose-tinted, soft-focus, lacklustre nature; you have to purr rather than prowl. If you make folk, you have to remove all the sinister poetry that's the black heart of the genre, and be pretty, winsome and weak. To shift units and make headlines, you have to be prepared to comply with the demands of a culture ruled by surface and style, and your persona has to be pliable.
But as the end of the year approaches, I find the landscape has changed. In recent weeks, the musician dominating the headlines has been the 15-stone, lesbian indie rocker Beth Ditto, frontwoman of the Gossip. She came tops in the NME's typically testosterone-heavy Cool List, thanks less to her music than to her fearlessness and sense of style. But musically, too, the artists pushing the boundaries of pop, soul and folk have been women.
Three of the best records of 2006 have been brought to us by Lily Allen, 21, a chart-topping, ska-sampling Londoner; Amy Winehouse, 23, a filthy-mouthed, down-to-earth diva; and Joanna Newsom, a harp-playing American folk innovator. The albums were, respectively, Alright, Still; Back to Black; and Ys. Unusually in the formulaic, manufactured world of contemporary pop, all of them clocked up substantial sales without compromising the fierce originality of the artists.
I interviewed Allen shortly before her career went into overdrive. She was witty and bright and as sharp as a razor. We talked about her style: straight-talking pop songs about snappy young girls in everyday situations, influenced by calypso, New Orleans jazz and reggae. I loved "Knock 'Em Out", which showed you how to turn down a sleazy man with sass ("I've gotta go 'cos my house is on fire!/ I've got herpes! Err no, I've got syphilis!"). Then there was "Smile", the number-one smash of the summer, which laughed in the face of a horrible ex. "Everything's Just Wonderful" took the biscuit, though, sticking up two brightly nailed fingers at feeling down about your body ("I wanna be able to eat spaghetti bolognaise [sic]/and not feel bad about it for days and days and days").
What impressed me most about Allen was her sense of mission about her music. She wanted to write for young people, and for young girls especially; to cut through the crap they got served up regularly. She said with suitable candour, "I want to write music for them to go, 'Wow, that's what it's really like! They do take drugs and they do suck dicks and it's amazing!'" Five months later, after also making the top ten of the NME's cool list, she hit back at the magazine's editor, Conor McNicholas, when he said that the women on the list were "living proof that you can still rock a crowd when you're wearing stilettos".
"I mean how fucking patronising . . . is that all we are, stiletto-wearing people?" she wrote furiously on her MySpace blog. "Is that all he could say? . . . Wankers." Pop stars were supposed to stay silent or simper. They weren't supposed to swear, or smoke on stage, or slag off the great and the good of the music industry.
The British soul singer Amy Winehouse seems to share a similar goal. "Somebody said to me recently that listening to my music makes them think, 'This is what it's like to be young and living in London in 2006,'" she told me. "I'm so fucking proud that it's achieved that. When I listen to the Shangri-Las, it's 1964 and I'm young in America. When I listen to the Specials, it's 1980 and I'm young in London. That's all I've ever wanted to do."
Winehouse's 2003 debut album, Frank, showcased a voice belly-deep with soul as well as rage, showing contempt for footballers' wives, bad lovers and weak-hearted men. Back to Black is even better: an astonishing soul record, soaking up the sounds of Motown and 1960s girl groups and spitting them back with panache, glamour and a contemporary twist. To a brass and drum backbeat, "You Know I'm No Good" includes references to "chips and pitta", "carpet burns", and a brilliant description of a boyfriend guessing that his girl's done the dirty ("You say, 'Why did you do it with him today?'/and sniff me out like I was Tanqueray"). The allusions to alcohol are everywhere, and Winehouse has admitted they have personal weight; besides a drunken appearance on Charlotte Church's Channel 4 chat show, her struggle with (and ultimate rejection of) rehab has helped her become a tabloid gossip regular. She has relaxed slightly now, but is still an open book about her experiences, and she has drawn on these demons in her dark and compelling work.
Next to these loose-lipped women, Joanna Newsom looks as if she hails from another world entirely. A fresh-faced 24-year-old from northern California, her recent album Ys is a 55-minute orchestral folk epic consisting of five very long, ghostly songs. It tells a story about meteorites, tugboats, sparrows and pharaohs. Newsom's 2004 debut, The Milk-Eyed Mender, set the precedent - featuring intricate, African-inspired harp-playing, a love for ancient poetry, and a distinctive voice that swooped and screeched and sailed - but Ys was undoubtedly the most eccentric and original record of the year. Like Allen and Winehouse, Newsom emerged from the shadows as an original talent, and a glorious leader for the genre in which she has been operating tirelessly. There was one very peculiar week this autumn when Ys was beating Take That's new album on Amazon presales.
Newsom says that during the creation of Ys she was "thinking about literature, really. I was thinking about how I've always been inspired by William Faulkner's The Sound and the Fury and its way of telling a story from five different viewpoints. I wanted to do this through music, and I wanted every meaning to matter, as well as every syllable and sound." We talked briefly about Nabokov and Hemingway and the composer Van Dyke Parks - who ended up orchestrating Ys and whose 1968 Song Cycle is a huge influence. When I marvelled at her ambition, Newsom laughed modestly. All she wanted to do, she said, was to make something new that came straight from the heart, and that people could get more and more from on each listen.
That is a good description of what all three of these women have done. Shining a light into the murky corners of mainstream music, they have offered up records that are bright and fresh. They have not toned down their characters, or compromised their obsessions - be they finding great sounds to sample, singing about great sex, or studying great novels. They want to communicate to their audience through their music, and they create worlds that stand out as distinctively theirs. And they are all sticking to this fiercely individual position in an industry that is too often concerned with repeating formulas. Allen, Winehouse and Newsom all have one charac teristic which is in short supply in our celebrity- obsessed culture: real personality.
Jude Rogers is reviews editor of The Word magazine