Changing the garde

A new breed of star showed us that big-name actors could be experimental too

Remove from film history the nobly suffering female face in close-up and there would be some yawning gaps where great cinema used to be. We're talking farewell to Carl Dreyer, cheerio Ingmar Bergman, TTFN John Cassa­vetes. And in any assessment of screen acting from the past ten years, it is still the women on the verge, or in the throes, of a nervous breakdown who dominate.

Think of Julianne Moore as the 1950s housewife in Far from Heaven (2002), excommunicated from the coffee-morning elite after fraterni­sing with her African-American gardener. Or Dina Korzun, a Hanna Schygulla for our time, torn between her shambolic husband and his resentful son in Forty Shades of Blue (2005). There was Lorraine Stanley, bruised but unbeatable, in London to Brighton (2006); Charlotte Gainsbourg bared her heart in I'm Not There (2007), a couple of years before baring her body in Antichrist. Melissa Leo, as a dirt-poor, nicotine-stained people-smuggler in Frozen River (2008), looked like death warmed up then left out overnight to go cold again. And please don't forget Nina Hoss falling apart in the corporate purgatory of Yella (2007), a movie that shares with Laurent Cantet's extraordinary L'emploi du temps (2001) the award for Most Far-Sighted Film of the Decade.

Tormented women were at the core of two of the pictures I most admired. François Ozon's Under the Sand starred Charlotte Rampling as a woman rebutting all evidence that her husband, who had vanished after wading into the sea, was dead. No less an authority than Bergman considered the film a masterpiece. Nicole Kidman explored the same close proximity between glacial conviction and devastating frailty in Jonathan Glazer's Birth, in which she played an Upper East Side widow confronted by a ten-year-old scamp claiming to be the reincarnation of her husband.

Both Under the Sand and Birth end on a beach with the camera maintaining a vigil as a bereft woman heads towards a future in which nothing, other than dependence on mood-stabilising medication, is certain. Something else connects these films: the pressure on their lead actors to carry in their faces information that can't be articulated in any other form. So, an actor expresses character through performance - big deal, right? But when Kidman, scrutinised by the camera at a Wagner concert, signals in her eyes an emotional breakthrough that is entirely interior, we are witnessing in practice Robert Altman's theory that actors can be auteurs, too - that they can provide an authorial presence in a movie as persuasively as any director.

The notion goes beyond acting: it can also be about how the performances shape our perception of the actor's body of work. Under the Sand at once refreshed and interrogated Rampling's arctic persona: it was a comeback movie of sorts. Birth, on the other hand, was a seductive part of Kidman's existing mission statement. Ten years ago, the world was only just starting to accept that she had a higher function than keeping Tom Cruise dry during rainstorms. With Eyes Wide Shut in 1999, she served notice that she wouldn't be restricted by her husband's narrow horizons, by playing a woman who wouldn't be restricted by her husband's narrow horizons. That was the beginning of the affair, the honeymoon period, the salad days. Not for Cruise and Kidman - they divorced in 2001, two days before the US release of Alejandro Amenábar's goosebump-tastic The Others - but for us.

While it was decrees absolute at dawn for the couple, our torrid romance with Kidman was fanned by a run of risky roles unbecoming of an A-list performer: The Others and Birthday Girl (both 2001), Lars von Trier's masterful Dogville (2003), Birth (2004) and Fur: an Imaginary Portrait of Diane Arbus (2006). (Imagine, say, Julia Roberts or Sandra Bullock in those films and you start to appreciate Kidman's radicalism.)

Kidman's career reflects the blurring in recent years between stars and character actors, mainstream and avant-garde. Russell Crowe seemed poised to follow Kidman off the straight-and-narrow, but watching him become increasingly inhibited by his power has been like witnessing the hardening of an artery. The trend of stars using box-office success as a springboard into the unknown is better represented by Viggo Mortensen, who got his first taste of stardom more than 15 years into his career, playing Ara­gorn in the Lord of the Rings trilogy. If you're looking for proof that commercial success can liberate an actor, take Mortensen's intuitive but intellectually rigorous character studies in two so-so David Cronenberg films, A History of Violence (2005) and Eastern Promises (2007). And would another performer of his stature have consented to a role as passive and complacent as the professor wooed by Nazism in last year's Good?

Perhaps only Johnny Depp. Ten years ago, Depp was an indie totem and heart-throb-in-waiting who hadn't quite flowered. But everything came together in 2003 when he flounced across the screen in the first instalment of Pirates of the Caribbean, lighter than air despite wearing three times his own weight in guyliner. He brought buoyancy to a blockbuster that was the epitome of corporate cinema (a task that almost defeated him in two lacklustre sequels). And his status has since insulated his idiosyncrasies rather than nullifying them. Unconsciously or otherwise, Kidman, Mortensen and Depp have insisted that the star can be an instrument of art rather than (or as well as) commerce, like superheroes dedicating their unearthly gifts to the service of good over evil.