Why political parties find it hard to learn from defeat

Cognitive and structural biases make it hard for party members to analyse what went wrong and plan their next steps.

NS

Sign Up

Get the New Statesman's Morning Call email.

"You may encounter many defeats," Maya Angelou told the critic Claudia Tate in 1989, "but you must not be defeated". In fact it was those moments of defeat, Angelou said, that had created in her "the vitality and the power to endure". Failure can be used as a source of wisdom, but without understanding, it can also be a potent fuel for further failure.

Labour’s worst general election result since 1935 has led to an outpouring of opinion on what the party needs to take away from the debacle. There is less discussion of how Labour should go about learning from defeat, but this is the question the party needs to answer first.

Each Labour defeat has been followed by a process of reflection. After the 2015 election, it set up a task force to "learn the lessons from defeat". After the party lost again in 2017, the result was framed as a victory. Both approaches may well have been hampered by the cognitive biases that prevent many organisations from learning from failure.  

Whatever their views and policies, political parties are generally conservative in the way they approach internal change. It is only when they face large-scale electoral losses that they embrace change, and even then they do so with one eye on the risks to the votes they managed to retain. Nor does transformation guarantee a party will win more votes. Last week's result is not the only example of this. One study of 23 countries found that voters tended to ignore the current policies of a party, and responded instead to the policies parties had put forward in the previous election. Another study of western European countries found that voters only noticed policy changes when they were accompanied by a change in leader.

One well-known cognitive bias that prevents political parties reflecting properly on defeat is the fundamental attribution error, which was first identified in 1967. This is our tendency to attribute successes to our ourselves, but to lay the blame for our failures on external factors. In reality, most political failures are a mixture of internal factors that the party could control, and external factors that it couldn't. But our tendency to ignore or understate our own role in our failures limits our understanding of what led to them. As with many cognitive biases, focusing on external causes of failure is common because it is the easiest response. It lets us off the hook, and it means we don’t have to change what we do.

The way parties are structured can also impede how they learn from failure, because this alters who the party listens to. One study of ten European countries between 1977 and 2012 found that parties with large activist bases tended to change their policies in response to what the base wanted. In contrast, parties controlled by their leaderships tended to change their policies in response to the preferences of middle-of-the-road voters. This suggests the Labour party, with its large base of activists, is most likely to change in response that activists within the party want, rather than voters outside the party. This pleases activists, but it could also mean the party is deaf to changing preferences in the electorate.

Further research has shown that this tendency to appease the faithful can also be exacerbated by defeat. A recent psychological experiment found that following the election of Donald Trump in 2016, Clinton voters who regulated their negative emotions after their candidate’s defeat often ended up feeling better - but were less likely to engage in activism. This suggests that discouraging reflection and stoking up rage offers short-term gains to a party, because it helps maintain a base of engaged activists. While it's understandable that a party reeling from defeat would want to shore up its base, this may also make it still less able to hear the concerns of voters over members.

There is mounting evidence that the most effective organisations in many sectors are those which can learn from failure, and there are practical steps which organisations have used to learn when things go wrong.

For political parties, there are four things that could help. First, the party needs to build a sense of "psychological safety" in its members. They must feel that they won’t be shouted down for sharing their failings. If members feel they are psychologically safe, they are more likely to be honest about what went wrong and open-minded in trying to fix it.

Second, members of a party need to be willing to recognise their own complicity in failure. This can be painful, but taking responsibility makes people more willing to change what they are doing rather than continuing with business as usual.

Third, a party needs to get people to step outside of the positions they feel comfortable with. There is a tendency for people to stick with one’s own preferred story of what went wrong. This makes us unwilling or unable to consider the perspectives of others. One way to get around this is by asking people who have different understandings of what went wrong to swap positions, and argue the other person’s case. By doing this, people often loosen up their assumptions and are more willing to see a wider range of problems. This process helps party members create new ways of understanding what went wrong and what can be done.  

Finally, it is unlikely a party will hit on one set of solutions to all the problems it faces. No matter how high the quality of the analysis, it is impossible to know whether solutions are correct until they have been tested against reality. A party seeking to recover from failure must experiment with a range of different solutions to the problems they have identified. What doesn’t work can be dropped, and what does work can be scaled up. But without an honest and open-minded approach, positive change becomes very difficult and further defeat all but inevitable.