The story of primary elections in America part one: An unfortunate evolution

US politicians spend more time on the campaign trail than in any comparable nation, and open primaries are a big reason why. But it hasn’t always been like this.

Sign Up

Get the New Statesman's Morning Call email.

The modern United States seemingly finds itself in a constant campaign for political office. Voters know the drill: the aftermath of a presidential election gives rise immediately to primary challenger speculation, dubiously leaked polling, and exploratory committees for the next one. 

Midterm elections give a lay of the land. For declarations of candidacy, you can pencil in a tightly choreographed home video or a fiery stump speech. Then, after 12 months of campaigning, six months of caucus and primary voting involving tens of millions of party-registered voters, the who’s-who of local, state and national party machines descend on convention-halls to rubber-stamp the people’s choice. 

Then – and only then – does the US presidential election get underway. 

But the bruising 2016 Democratic primaries and the questionable vetting abilities of their Republican counterparts cast doubt over the merits of an open primary system. Most comparable Western, liberal democracies aren’t so accommodating of constant electioneering. 

The British and Canadian elections run for 38 days and 36 days, respectively. Australia registers as one of the shorter campaign seasons at a minimum of 33 days and a constitutionally mandated maximum of 58. And despite some French parties gradually moving towards open primaries, their elections push the whole process out to a comparably short six months. The open primary system as known to America is, for the most part, non-existent in these jurisdictions.

This is why US politicians spend more time on the campaign-trail than their counterparts in almost any comparable democracy. In the information age, politicians asking for your money or your vote prove almost inescapable; more than just a nuisance. Concerns of democratic fatigue plague these marathon campaigns. 

Consider, too, what sort of personality this gruelling process attracts – will it be those who will spend their time actually governing, or those who seek the approval of others and revel in the proverbial knife-fight of polarised politics.

The gluttons for punishment among us might say that this drawn-out selection of candidates is the most democratic method for choosing our leaders. After all, voters not only choose who they vote for, but who they get to vote for. But candidate selection is about more than just election length; paradoxically, the experience of other countries shows that less democratic candidate selection processes might actually better shape our political class and strengthen democratic safeguards.

Despite a gradual shift away from caucus ballots to primary-like party leadership votes, Australia and the United Kingdom opt for closed, intra-party candidate selections for parliamentary seats. Referred to as a preselection, candidates in this process are chosen internally by select party members with voting rights. 

To prevent interference, voting rights are typically allotted to a few hundred fee-paying members in each electorate who have been members of the party for a period of time longer than a uniform probationary period. If the candidate is any good, local party members will know them personally, as a friend and equal. Yet, parties in Australia and the United Kingdom are increasingly feeling the need to democratise, beginning with how they elect a party leader.

In these liberal democracies, the view that political parties are private organisations is also more prevalent, likely helped by the fact that taxpayers don’t pay to facilitate candidate selection as they do in the United States. The preselection system is designed to reward activists who invest time and money in the party. 

Most importantly, a higher bar for membership and the closed nature of these elections creates the unofficial role of party powerbrokers, who derive influence based upon on locality or ideology. Their role in ensuring a responsible candidate selection is important. Caucus ballots represent another check on the electorate, in which elected members of the party caucus have a role in electing their party leader much as they do for the speaker in the US House of Representatives. 

And while I am comparing candidate selection across different systems of government in Westminster and Washington, the United States could just as easily adopt a closed-party system – in fact, it has done so before.

How we got here

Despite dominating modern American politics, the primary election as we know it is a rather recent development. Only fully realized in the 1980 Reagan electoral cycle, the progression of candidate selection in the United States has broadly taken three forms, according to Bruce J. Schulman, the William E. Huntington Professor of History at Boston University: nomination by the statehouse or congressional caucus, nomination by a convention of party bosses, and open caucus and primary elections.

In a system which bears similarities to modern-day preselection in Westminster-style parliamentary democracies, the American republic’s earliest method of candidate selection empowered elected representatives to nominate their party’s presidential candidate. The first nominating conventions of party delegates were introduced as the country moved toward the mass, catch-all parties of the Jacksonian Era which would see two parties dominate the entire ideological spectrum. 

Still, this system enfranchised relatively few electors. Schulman advises that in this model “there [were] party caucuses and state party conventions that selected delegates, but in general the delegates [were] not pledged to any candidate and often they [were] controlled by … the big state or city leader, [who would] have a block of delegates.” 

According to Schulman, these conventions were characterised by states “horse-trading” delegates in return for cabinet posts and policy outcomes. Sometimes this internal deliberation could last for up to two weeks.

Despite the concept of open primaries first gaining traction in the early 1900s, for about 70 years just 15 states facilitated the elections. Schulman says “as late as 1968 there were only 15 primaries, and those primaries only accounted for a third of the national nominating convention.” 

Primaries had little-to-no decisive impact on nomination outcomes. In furthering this point, he cites Hubert Humphrey’s 1968 Democratic presidential nomination. Humphrey didn’t win, nor did he even contest, any of the 15 open primaries. But 1968 did mark a turning point for the US electoral system.

Humphrey’s nominating convention was marred by widespread protests and bloodshed which took place outside on the streets of Chicago. Conflict between demonstrators and police, under the control of Democratic powerbroker and Mayor Richard Daley, exemplified the split between "new politics" constituencies and the entrenched party bosses. 

The increasing enfranchisement of women, African-Americans and young people activated in opposition to the Vietnam war posed a challenge to the politics-as-usual nomination of the sitting vice-president. Despite their newfound voice, these voters were militating in favour of causes much the same as those which guided the early 20th Century progressive reform movement. 

Fighting “what they viewed as corruption in politics,” Schulman claims the progressive reform movement wanted to “not only weaken the power of party bosses but weaken the power of plutocrats [also].” 60 years on, this battle against the establishment was picked up again by a movement more diverse than anything America had yet seen.

In response, the DNC created the McGovern-Fraser Commission to reform candidate selection by expanding open primaries. Delegates would be bound by their state’s primary election. As a consequence, the 1972 nomination saw George McGovern translate success in the primaries into votes on the convention floor – to the well-publicised chagrin of the party old guard. 

The Republicans went on to match the selection reforms, but in the ensuing decade the Democratic establishment would reclaim some influence through the introduction of superdelegates. In late 2018, the DNC opted to remove the influence of party officials as delegates unless in the case of a contested convention.

The evolution of candidate selection in America is fundamentally about tension between grassroots activists and party elites, between a more participatory democracy or closed-door politicking. But did these progressive elements and the seismic electoral changes they brought do more harm than good?

This is part one of a three-part series. Part two, which compares America’s primary system with that in other developed democracies like the UK, will be released next week. Part three, which considers potential solutions, will be released the week after.

Corbin Duncan studies Government and History at Harvard University and formerly worked as a political advisor in Australia. He tweets @DuncanCorbin.