No one who watches American movies regularly is ever likely to bemoan a lack of father/son stories. At times it can seem that the entire industry is run by studio executives greenlighting films about men who are torn, as they are, between their careers and their sons’ little league games. The nadir of this psychotherapy-as-cinema sub-genre was surely Field of Dreams, the 1989 Y-chromosome weepie starring Kevin Costner as a farmer who got to play baseball with the ghost of the father he had taken for granted. As with many of the malignant elements in American cinema – the reverence towards opening weekend takings, the supremacy of the blockbuster, the career of Michael Bay – it would be possible to blame this whole voyage-round-my-father cinematic movement on Steven Spielberg.

Possible but also simplistic. Spielberg may not be a profound artist but he is a painstaking one, and his ongoing exploration of the father- son dynamic defines him as much as any of his staggering visual coups. (Unlike his imitators, he doesn’t exclude women from the equation: for all that it is motivated by the absence of the father, E.T. the Extra-Terrestrial features one of the most sympathetic portraits of motherhood in all cinema.) But from Spielberg’s modest debut, Duel (a one-character, 90-minute thriller made for TV in 1971 and released theatrically), to his latest film, Lincoln (nearly twice the length, with around 150 speaking parts), the concept of the father has been at the core of his vision of the world.

No journalistic spadework is required to join up the dots to the director’s own life. “I missed my dad a lot growing up, even though we were together as a family,” he said recently. “My dad was really a workaholic. And he was always working.” Rarely can a filmmaking career have been devoted so singlemindedly to addressing such a schism. Fathers in Spielberg are in prison (The Sugarland Express), away at sea (Jaws) or estranged physically (E.T.) and emotionally (Indiana Jones and the Last Crusade); they are fiscally vulnerable (Catch Me If You Can); their obsessions can jeopardise the family (Close Encounters of the Third Kind, Munich). Movies such as Empire of the Sun, Hook, Saving Private Ryan and A.I. – Artificial Intelligence are as littered with lost sons as any orphanage, so it can only have been a matter of time before Spielberg alighted on a reliable father like Abraham Lincoln to assuage the traumas of his own past.

Whether the making of Lincoln helps Spielberg adequately to process the pain of not having his father in the bleachers is a matter for him and his therapist. But there must have been a likelihood that Lincoln would fulfil the same function for the subject of slavery that Schindler’s List did for the Holocaust: that is, to act as a minor balm to a history of deep and resonant suffering. “Think that’s about the Holocaust?” Stanley Kubrick once asked witheringly of Spielberg’s Oscarwinning movie. “That was about success, wasn’t it? The Holocaust is about six million people who get killed. Schindler’s List is about 600 who don’t.”

Lincoln does provide a measure of comfort but it also contains a larger share of complexities than we have been accustomed to finding in late-period Spielberg. For example, it could be viewed superficially as inspirational in the modern context of President Obama’s tribulations at home: with the House of Representatives bitterly divided, what could be more uplifting than the spectacle of opposing sides united to vanquish a gross injustice by passing the Thirteenth Amendment, outlawing slavery? On the other hand, the stringpulling, mischief-making and horse-trading entered into by Lincoln (Daniel Day-Lewis), his secretary of state William Seward (David Straithairn), and their advocates – notably a trio of wily Republican vote-snaggers (John Hawkes, Tim Blake Nelson, James Spader) – is hardly designed to have audiences saluting the US flag. At its funniest, Lincoln suggests a backstage political caper – The Thick of It with stovepipe hats and tumbleweed sideburns.

Advance enthusiasm had suggested that the movie would be the director’s chef d’oeuvre. Well, it’s certainly someone’s but it is surely the screenwriter, Tony Kushner (the playwright best known for his Pulitzer-winning Angels in America), who has brought the film its unique structural and linguistic strengths. A single-film biopic of Abraham Lincoln would have been foolhardy, so Kushner has created a fine-grained procedural drama, a portrait of the man through the prism of the battle in which his mettle was tested most fiercely – the sort of speculative reconstruction made bankable in recent years by the screenwriter Peter Morgan (The Deal, The Queen, Frost/Nixon).

The speculative element has been reduced considerably by Kushner’s fidelity to one section of the historian Doris Kearns Goodwin’s book Team of Rivals: the Political Genius of Abraham Lincoln. Kushner concentrates on the first four months of 1865, the last of the five years examined in depth by Goodwin, in which Lincoln invested all his energies in getting the Thirteenth Amendment passed as the civil war spluttered on.

Political nerds will experience a wave of delight as they realise that the film is going to scrutinise forensically the process by which Democrat opinion was turned and each individual vote secured. In this Lincoln most resembles Milk, Gus Van Sant’s 2008 movie about the election of America’s first openly gay male politician, Harvey Milk. Viewers hoping for a disco-era camp-a-thon might have been flummoxed by that film’s preoccupation with issues such as voting boundaries and campaign strategies. Similarly, Kushner’s screenplay is a tapestry arrived at one apparently inconsequential stitch at a time.

The only other notable film about the sixteenth president – here I must disappoint regretfully any admirers of last year’s Abraham Lincoln: Vampire Hunter – is John Ford’s 1939 Young Mr Lincoln. Like Spielberg and Kushner’s film, Young Mr Lincoln followed a foreshortened narrative: it concentrated on its subject’s early years and found in his time as a lawyer some harbingers of triumphs to come.

There is a surprising continuity between Henry Fonda’s alert and witty performance in Ford’s film, and Daniel Day-Lewis’s stately turn in Lincoln. For all their differences in methodology, there’s a baked-in wisdom and joyfulness that runs through both actors’ portrayals. Playing the younger man, Fonda was bound to have the edge in spryness, but Day-Lewis’s Lincoln is an intellectually ravenous figure who savours ideas, words and stories (especially his own). This is a performance of ruminative stillness but one that reveals the mental sparks flying even in moments of repose. He’s playing a man of just 56 years old (Day-Lewis himself is only a year younger) and he advances slowly, majestically rather than falteringly, as though he is thinking through the ramifications not only of each political move but of putting one foot in front of the other. It’s all the more shocking, then, when he raises his voice, or lashes out at his son, Robert (Joseph Gordon-Levitt), whom he has forbidden from enlisting. The slap represents a rare moment of Lincoln losing control, of reaching for a weapon other than rhetoric or anecdote, but it also serves to remind us of the lightning reflexes beneath the meditative calm.

Lincoln begins with an intimately gruelling scene of civil war combat: punches are thrown, bayonets jabbed, a man is drowned in a muddy puddle. It’s a remake in miniature of the Omaha Beach sequence from Saving Private Ryan, which subverted for many viewers what the experience of a Spielberg movie could be. Thereafter, the fury and frenzy of Lincoln is entirely verbal. In a film that values the back-room mechanics of politics as much as the progress that results, it’s necessary to have actors who know the nutritional value of a hearty script.

Among a fine supporting cast, Tommy Lee Jones gives the most passionate performance of his career as Thaddeus Stevens, the Republican whose personal fervour for the abolitionist cause proves inflammatory to his opponents. Only Sally Field, as the first lady, Mary Todd Lincoln, does an inadequate job: even Mary’s abrasive stand-off with Stevens at a dinner party can’t stop this actor drawing from her well of self-adoring tomboy toughness.

It seems almost by-the-by to praise Day-Lewis, so accustomed have we become to his landmark status among modern screen acting, but he is genuinely mesmerising, not least in his concentration and his delivery: he brings a lolling looseness to his lines so they sound like they’ve only just occurred to him. A scene depicting Lincoln meeting wounded soldiers in hospital is rendered moving by Day-Lewis’s unforced affability (“Tell me your names as I go past – I like to know who I’m talking to,” he says in a warming burr). His charisma is quietly dazzling; he makes you understand why the soldiers would get a kick simply from standing in his shadow.

Day-Lewis, Spielberg and Kushner conspire together in an overhaul of Lincoln as icon, breaking apart what we know of the man and building him up again over the course of the movie. In that nocturnal battlefield scene near the start, Lincoln sits with his back to us in semi-darkness as a group of soldiers gather round to meet him. Kushner had the tremendous idea of dismantling Lincoln’s speeches so that we never hear him deliver any of them (he is seen only once, briefly, at the podium). Instead, one of the adoring soldiers recites part of the Gettysburg Address back to Lincoln, who listens humbly.

Just as Jane Campion put some of Keats’s poetry in the mouths of children in Bright Star, Kushner and Spielberg remake the familiar. The cinematography by Spielberg’s regular collaborator, Janusz Kaminski, has a magisterial grandeur – all architectural greys and steely blues burnished with pockets of warmth – but the scenes themselves can be oddly informal, whether it’s Lincoln describing a dream to Mary or lying on the floor in his stockinged feet.

In its treatment of slavery, Lincoln improves immeasurably on Spielberg’s last brush with the subject – the 1997 courtroom drama Amistad, an African-American story compromised unnecessarily for white audiences. Black characters don’t make much of a showing in Lincoln but their presence resonates beyond individual scenes. I particularly liked Lincoln’s fleeting conversation with Mary’s maid (Gloria Reuben) concerning his feelings towards her race. He gives an apologetic half-smile and admits: “I don’t know you. I assume I’ll get used to you.”

The high-water mark in the treatment of slavery in mainstream cinema has to be Ang Lee’s 1998 Ride with the Devil: what appears to be a white man’s story is eclipsed in its final moments by the quest of a black slave (Jeffrey Wright) to find the family from whom he had been separated. That was poetic justice expressed in film language, as a marginalised character gained control of the narrative. Lincoln doesn’t have anything quite so radical but it comes a close second with a point-of-view shot that puts the camera behind African-American eyes and makes the president’s butler the last person in the movie to see him alive.

Lincoln is released on 25 January