The Bot Wars: why you can never buy concert tickets online

Enterprising programmers are creating bots that can reserve, and in some cases buy, everything from restaurant tables to eBay goods before humans can even get a look in. Where will the bot wars end?

Diogo Monica was frustrated. All he wanted was to book a table at his favourite restaurant, but it seemed to be an impossible task.

State Bird Provisions, in the Fillmore district of San Francisco, was fully booked for 60 days in advance, and had been ever since last August, when Bon Appétit magazine named it the best new restaurant in the country.

No matter how many times he returned to the website, the answer was always the same: “No reservations are currently available. As tables become available, they will be shown here.”

Sure, the crispy spiced quail is “unparalleled”; yes, the rose-geranium ice cream is “a singular delight”; but could the promise of such culinary gems really rouse so many to try, every single day, that a table became literally unbookable?

Diogo wasn’t the only one having problems, a myriad of online complaints attested to that. “It’s the hardest place to get a seat in the city right now, full stop," said one. “Don’t even bother,” said another. “Tried every day last week = 0 bookings.” 

Where could all the reservations be going? Were they even going up at all?

Unlike most customers, however, Diogo had another weapon in his arsenal. As a computer programmer, currently working for Twitter founder Jack Dorsey's new venture Square, he had the skills to build a bot.

“A bot is essentially a computer program that automates the process that a human would go through when making a reservations,” Diogo explained via email. This bot would scan State Bird Provisions’ reservation page at regular intervals, emailing him every time it was updated. “I set it to run once every minute.”

At first he had some success. The resulting flurry of emails showed him that reservations really were being released, albeit at four in the morning. Other people were on to this too: each day, without exception, every table would be snapped up within the hour. Armed with this knowledge – and the occasional tip off about a last-minute cancellation – Diogo began to get reservations again.

It wasn’t fair – that he knew. Other fans without his technical skills were still going without a table. But it was a relief to have some success at last after expending so much time and energy on the project. 

But soon, even armed with this insider knowledge, the tables were being grabbed from under him – even the unexpected, irregular cancellations. “The moment they became available they would be gone in the time it took for the website to load.”

He realised that his bot was up against something more powerful. Someone else – or maybe several others – had created a bot that could both watch for and then reserve the available tables all by themselves.

“You fight fire with fire,” he said. “So I decided to fight back, creating my own.”

And so he did. The days when Diogo couldn’t get a table for love nor money are now a distant memory. His bot is so good at its job that sometimes he gets two or three reservations in his name in the same day – he thinks he’s now been to the restaurant “around 15 times”. But he doesn’t expect to stay top for long. If he wants to keep getting tables at State Bird Provision, he’ll have to constantly modify his bot as rivals are adapted to undercut him.

Diogo wrote about the "bot wars" on his website – releasing the code for anyone who wanted, in a bid to “level the playing field” for the less technically-minded.

Soon his competitors were coming out of the woodwork too, including Anil Bridgpal, a former Comcast software engineer.

News spread quickly among the foodie community: his tale of automatons, of reservations made in superhuman speed, rang true with many who had spent fruitless hours refreshing and refreshing the reservation page.

“Weird things go on,” agreed one thwarted diner, who had met with precisely the same problem with a restaurant 60 miles away. “I submitted my reservation within two-tenths of a second after midnight and was told I was too early. Within three seconds I resubmitted and the message said nothing was available. This is not the first time this has happened to me.”

It wasn’t just him, Diogo realised; it wasn’t just this restaurant.

***

Diogo’s dispatch from the frontlines of the new bot wars provided a rare insight into an underground community of unknown, unseen programmers who compete to build the fastest, most powerful bots in a virtual arms race.

Just as high frequency trading, via automated software, took over the financial markets in the early 2000s, the use of bots is a technique that is increasingly coming to dominate online sales of all stripes.

Diogo admits to deploying bots himself not only for dinner reservations, but to secure cinema tickets on busy opening nights and the cheapest flights to visit his family in Portugal. Anil knows of bots for “concert tickets, camping grounds and marathon registrations”. Such programs “are pretty easy to for any developer to write,” he told me.

Yet despite their ubiquity, in hacker circles there is little discussion of the ethics of bot use. There are no gentleman’s agreements to abstain from silicon wizardry when fighting it out with the general public. When there’s an online race, it’s every man for himself – and no one said the race would be fair.

While this might still be only a mild irritation when booking restaurants – where bot use is confined to only the most popular places – it already poses an enormous problem for the music industry. Online ticket sales for concerts are increasingly dogged by accusations of foul play from fans who feel they have been leapfrogged by so-called “scalpers” who buy up dozens of tickets as soon as sales open, then sell them immediately on, for a profit.

When Beyoncé announced an eleven-date UK leg of her 2013 “Mrs Carter” tour, fans knew that competition for the tickets would be fierce. So, on the morning of the 23 February, before the sun had risen, TV researcher Gemma Meredith was ready at her computer, the booking page on her laptop screen and her credit card on the table in front of her.

By 9.29am, she was refreshing the page repeatedly, jostling for first place in the queue. At 9.30 exactly she had been redirected to a "virtual waiting room" – then minutes later, the verdict: “Tickets could not be found. Please try again.”

So she did. Again, then again. Twelve minutes later, while she was still waiting, Ticketmaster announced that every single ticket had been sold. “It was incredibly frustrating,” she said. “Who was buying these tickets faster than I could reload the page? I don’t see what else I could have done.”

Across London, events manager Philippa Brady was in exactly the same position. “Me and a friend were both trying on two different computers – and neither of us got anything. I'm usually pretty good at getting tickets, but the Beyoncé sale was a joke.”

Even Olympic swimmer Rebecca Adlington had no luck, telling her followers on Twitter: “Three computers, refreshing like crazy but no Beyonce tickets! Hope anyone else trying has had more success!”

Yet within moments of the tickets selling out, touts were already offering them on eBay for as much as £1,000 for a standing ticket. “"There were so many, someone was obviously making a lot of money. But what I didn't understand was how they had bought so many when I couldn't even get one," said Gemma.

Ticketmaster, like all the other major ticket outlets, is in the midst of a huge battle with tout-controlled bots. During the Beyoncé sale alone, it thwarted 120,000 ticket requests from bots, and the number that got round the defences cannot be known.

A spokesman for the company said: “Attempts are made during every busy on sale to purchase tickets unfairly using bots. This is an arms race between the technology companies like us and the individuals who create bots. These people are continually innovating and refining bots to make them harder to identify. 

“Ticketmaster is committed to investing in technology that thwarts these bots and we will continue to work hard to ensure that tickets go directly into the hands of genuine fans.”

In the US, bots are thought to account for 90 per cent of traffic to the Ticketmaster website, and 60 per cent of ticket sales to some of the most desirable events. Equivalent statistics were not available for the UK, the spokesman said.

Ticket scalpers can be found advertising online, seeking programmers to develop ever more advanced software. One advert, now ended but still visible online, requests a program to “automatically search for tickets to events on Ticketmaster.com the moment they go on sale”. It must be able to “solve CAPTCHA screens with 75 to 100 per cent accuracy” and evade “cookie or IP conflicts that will lead to Ticketmaster blocking me”. A freelancer based in India was paid $450 for the job.

Indeed, anyone with the money can now buy ready-made bots from hackers "off the shelf". Websites like ticketbots.net offer, for a price, ticket-buying bots to suit a number of different sites - including that of Ticketmaster (£645), eBay-owned Stubhub (£385) and even the Royal Albert Hall (£490).

“Grabs tickets and hold for you instantly,” the Royal Albert Hall bot promises. “Specify section and/or row. . . automatically purchases tickets as soon as they are found.” It is the modern day equivalent of paying someone else to queue. The botmaker declined to reveal how many he had sold.

Yet despite this burgeoning market, some benevolent developers have been willing to use their skills for the greater good.  Adam Naisbitt, an entrepreneur from Milton Keynes, was so frustrated by the difficulties of buying tickets to the London Olympics that he created a bot to notify him when seats became available. But instead of keeping it to himself, he released the information on Twitter, under the @2012TicketAlert username.

“The response was overwhelming,” he said. “We had over 100,000 followers, reached over 2.5m people, helped thousands get tickets and raised over £2,000 for the Olympic Foundation.”

The popularity of his feed, and the success of bots more generally, comes down to the fact that these (often quite simple) automated programs can surpass human abilities in several ways. They are faster, making instantaneous ticket choices, or lodging eBay bids in the final milliseconds of a sale, and capable of making more rational decisions that humans – which has led to them being marketed for use in online gambling. “They do not make mistakes or stake more money because they believe a particular team or horse will win,” said Rade Jaramaz, director of a British-based company that develops bots for use on the Betfair exchange.

Some bots are simply successful because they don’t get bored, and will repeat the same action, again and again, without a drop in accuracy or motivation.

For this reason, many hackers built their first bots for use in computer games. Online games often feature a requirement to complete repetitive tasks to earn virtual wealth, such as mining for gold – the profits of which can then be traded for in-game bonuses or even real-life money. By writing a program to automate this process, a player can benefit from hours of mining without having to do any of the boring work themselves.

Just as in real life, the effect of the bots was to skew the market. For those who use the bots, your task becomes easy – whether that be mining virtual gold or buying real, sought-after concert tickets. For those that don’t, well, they are at an enormous disadvantage.

In many of those games, the same pattern emerged. As more and more bots came online, the situation spiralled out of control. Inflation skyrocketed as the gold-rich bot-owners priced out everyone else. Some games, like Diablo III, saw a total collapse of their virtual economy.

In the end, for the rule-abiding players, it stopped being fun. It stopped being fair. So they stopped playing altogether.

Such is the challenge facing the online sales industry. Every time a wall is built to keep out the bots, the hackers will be programming around it, popping up behind it, whack-a-mole style. But unchecked, the bots will cause havoc, undercutting the real fans, who will be forced to buy from touts, at spiralling prices.

Too many disappointments will see customers turning away altogether. Or, perhaps, they will buy their own software, hoping to beat the scalpers at their own game.

And thus the bot wars shall escalate. 

Is it fair that programmers can build bots to buy things you want? Photo: CJ Isherwood on Flickr, via Creative Commons

Cal Flyn is a freelance journalist, who writes for the Sunday Times, New Statesman and others. Find more of her work at www.calflyn.com and her Twitter handle is @calflyn.

Getty Images
Show Hide image

Jeremy Corbyn fights to the last in the war both sides knew would come

It is the Labour leader's sense of obligation to his supporters that sustains him. 

It was at 4.36pm on 28 June that Jeremy Corbyn officially lost the confidence of his MPs. “Following the ballot conducted today, the Parliamentary Labour Party has accepted the following motion,” an email from the party’s press office announced. “That this PLP has no confidence in Jeremy Corbyn as Leader of the Parliamentary ­Labour Party.”

The totals for each side were not revealed, but one of the rebels immediately phoned me with the numbers: 176 for and 40 against (with four spoilt ballots and 13 abstentions). Eighty one per cent of those who voted opposed Corbyn. Almost any other leader would have resigned at this point. In November 1990, Margaret Thatcher departed after 204 of the Conservatives’ 372 MPs voted for her against Michael Heseltine – proportionally more than twice the support Corbyn enjoyed. By the time of the ballot, nearly 50 of the Labour leader’s frontbenchers had resigned, leaving him incapable of assembling a full team.

But in a statement issued 20 minutes after the result, Corbyn declared that he would not “betray” his supporters by resigning. If he was unfazed by losing the confidence of the PLP it was because he never enjoyed it to begin with. As hostility to Corbyn's leadership intensified, his team had been anticipating a coup attempt for months. Momentum, the left-wing activist group, privately told its foot soldiers to prepare for another leadership contest this summer. 

Corbyn entered office in September 2015 with a formidable mandate from members/supporters and a feeble one from MPs. As few as 14 of the latter (6 per cent) voted for him, compared to 251,417 of the former (59.5 per cent). His first shadow cabinet contained just three backers: John McDonnell, Diane Abbott and Jon Trickett. Almost all of those who chose to serve under Corbyn regarded him as unelectable and incapable of leading an effective opposition.

Andy Burnham, one of the few who has not resigned from the shadow cabinet, told an undercover Sun journalist posing as a donor: “Privately, it is a disaster for the Labour Party . . . I think the public will think Labour has given up on ever being a government again.” But Burnham and others accepted posts in the hope that they could “make it work”. They aspired to moderate Corbyn’s stances and to form a credible parliamentary outfit.

Yet as his tenure went on, they found it ever harder to maintain this rationale. Rather than adopting a pragmatic “soft left” programme, Corbyn continued to advocate positions such as unilateral nuclear disarmament and the return of the Falkland Islands to Argentina. He sacked frontbenchers and threatened others in a reshuffle following the “free vote” on Syrian intervention. His speeches in the Commons were rambling and often obtuse (as on the occasion when he failed to mention Iain Duncan Smith’s resignation). He responded sluggishly to warnings of increasing anti-Semitism in the party. On 5 May, he became the first opposition leader since 1985 to lose council seats in a non-general-election year.

And yet, as of early June, just one shadow cabinet member (the former shadow attorney general Catherine McKinnell) had resigned. For months, increasingly exasperated backbenchers urged the front bench to “show some leadership” and revolt against Corbyn. It was only after the UK voted to leave the EU on 23 June that the “make it work” faction concluded it could not work.

Corbyn’s performance during the referendum campaign was regarded as not merely unenthusiastic, but obstructive. “You cancelled meetings,” Alan Johnson complained at the PLP meeting on 27 June. “No one from your office turned up to steering meetings. I would have been really grateful if anyone in your office had rung up to tell us what we were doing wrong.” On 29 June, in a letter to Corbyn leaked to the New Statesman, Labour's MEPs demanded his resignation and declared: "We were simply astounded that on Friday morning, as news of the result sank in, an official Labour briefing document promoted the work of Kate Hoey and Gisela Stuart for the Leave campaign." 

Some even suspect Corbyn of having voted for Brexit in the privacy of the polling booth. Chris Bryant, who resigned as shadow leader of the Commons on 26 June, said that he “didn’t answer” when pressed. A Facebook post by Keith Veness, a close friend and his former agent (for a decade), stated that “JC should have come out openly [emphasis mine] for Brexit”. Corbyn is also alleged to have told Martin Waplington, a computer programmer, that he was “going to vote to leave” when the two spoke at Meson Don Felipe, a tapas restaurant in Waterloo, south London.

 

***

For many MPs, the country’s backing for Brexit was all too predictable. They had long warned that Labour’s low profile (many of its voters did not know the party’s stance) and Corbyn’s unashamed support for free movement was strengthening Leave in Labour’s heartlands. On the morning of 24 June the rebels resolved to challenge Corbyn by submitting shadow cabinet resignations, tabling a vote of no confidence and, if he refused to relent, launching a leadership contest.

The first move was triggered unexpectedly early when the leader sacked Hilary Benn as shadow foreign secretary after several Sunday papers reported that he was set to demand Corbyn’s resignation. The spontaneous nature of the rebellion was demonstrated by Tom Watson’s presence at Glastonbury. The deputy leader’s mafioso reputation and personal mandate was such that he had long been viewed as the greatest threat to Corbyn. But Watson, who Snapchatted himself at a silent disco at 4am, was caught unawares. The next morning he was photographed in shorts at Castle Cary railway station in Somerset, transfixed by his phone as he made a hasty return to London.

Watson’s tacit support was regarded by the rebels as essential to the success of any revolt. When he met Corbyn on the morning of 27 June, opening with a five-minute chat about Glastonbury, he did not formally demand his resignation. But Watson warned him that he had lost all “authority” within the PLP and would face a “bruising” leadership election. The rolling resignations were co-ordinated by Conor McGinn, an opposition whip and close ally of the deputy leader.

“Blairites out!” chanted Momentum protesters in Parliament Square on 27 June. But the depth and breadth of the resignations made it impossible for Corbyn’s team to ­frame this as a factional revolt. A defining moment came when the Labour leader lost the “soft left” from the shadow cabinet: Owen Smith, Lisa Nandy, John Healey, Kate Green and Nia Griffith.

“Lisa and Owen were both in tears coming out of the meeting,” a source told me. “McDonnell just barged into what was meant to be a private meeting with Jeremy about the way forward . . . He sat on the table in front of JC and spoke across him, saying they would fight everyone. It became obvious that he had no interest in holding the party together.”

At 6pm on 27 June, the schism between MPs and members was dramatically symbolised. Inside Committee Room 14 in the Houses of Parliament, MPs rose repeatedly to demand Corbyn’s resignation. Meanwhile, in Parliament Square, thousands of hastily assembled activists attended an emergency “Keep Corbyn” demonstration.

The hitherto uncritical MP for Stoke-on-Trent South, Robert Flello (“Check my Twitter feed”), was the first to speak at the PLP, declaring: “For your sake, but most importantly for the people who need a Labour government, do the decent thing.” Another MP, the left-leaning former shadow minister Chris Matheson, told Corbyn: “I agree with your politics but you aren’t a leader. I’ve done something you’ve never done: won a seat from the Tories.” Johnson, the head of Labour’s vanquished pro-EU campaign, said: “I’ll take my responsibility, you need to take yours.” When the new shadow energy secretary, Barry Gardiner, defended the leader he was booed for defying the convention that shadow cabinet members remain silent. The meeting ended with not one MP speaking against the motion of no confidence in Corbyn tabled by Margaret Hodge and Ann Coffey. 

Out in Parliament Square, the Momentum gathering was addressed by McDonnell and then by Corbyn. “Those wanting to change the Labour Party leadership will have to take part in a democratic election,” the Labour leader tweeted afterwards. He reaffirmed the message the following day. “The obligation that Jeremy feels to those who voted for him is what sustains him,” a senior ally told me.

Under the electoral system introduced by Ed Miliband in 2014, the members are sovereign. The expectation is that they will again determine Labour’s fate. Though the party’s lawyers have ruled that Corbyn would not automatically be on the ballot (he would be required to win 50 MP/MEP nominations), few believe that he will ultimately be kept off. Legal advice to the leader's office has suggested that he would qualify as of right. It is Labour's National Executive Commitee which will settle the matter. Its pivotal role was highlighted when Ken Livingstone stood down on the afternoon of 29 June ("as my suspension stops me from attending") in favour of Corbyn supporter Darren Williams. 

After fielding six candidates in last year’s leadership election (three of whom made the ballot), Corbyn’s opponents were determined to unite around one. At the time of writing, Angela Eagle, who resigned as shadow first secretary of state, appeared the likeliest challenger. The former cabinet minister, who impressed when deputising for Corbyn at PMQs, has been preparing to stand for months.

MPs speak of her enjoying support "across the span" of the Parliamentary Labour Party, from the "soft left" to "moderates" to "Blairites". A source told me: "It is no surprise that colleagues are turning to her. She is very much considered a tough, Angela Merkel-type figure who can lead the party through this difficult period." As a trained economist (she served as exchequer secretary to the Treasury under Gordon Brown), Eagle is regarded as having the skills required for a post-Brexit world. She would also give Labour its first permanent female leader in its 116-year history. But some doubt her popular appeal. "Fourth in the deputy leadership to first in the leadership in 10 months is a big challenge," one senior MP noted. 

Owen Smith, the former shadow work and pensions secretary, was also said by sources to be contemplating a bid. The Welsh MP revealed his leadership ambitions to the New Statesman in January, telling me: "I think any politician who comes into this to want to try and change the world for the better, starting with their own patch and working outwards, I think they’re either in the wrong game or fibbing if they don’t say, 'if you had the opportunity to be in charge and put in place your vision for a better Britain would you take it?' Yeh, of course, it would be an incredible honour and privilege to be able to do that." Smith would offer himself as a competent, soft left alternative to Corbyn. Unlike Eagle, he did not vote for air strikes against Syria last year, regarded by some MPs as a precondition for success. As a minister in the last Labour government, Eagle also voted for the Iraq war, an issue that will return to the fore with the publication of the Chilcot report on 6 July. 

Watson, viewed by some MPs as the most formidable potential challenger, ruled himself out of the contest on the evening of 29 June. As deputy leader, he has long regarded it as his duty to preserve party unity above all else. A challenge to Corbyn, pitting him against most current members (including a significant number of those who voted for him), would unavoidably conflict with this role.

Watson's hope was that Corbyn could be persuaded to resign, leaving him as the automatic interim leader. But his final attempt to do so, which followed appeals from former leaders Gordon Brown, Ed Miliband, Harriet Harman and Margaret Beckett, foundered. Watson was described by sources as having been "blocked" from having a "one-on-one conversation" with Corbyn about his future. Rumours swirled among MPs that the Labour leader wanted to cave in but was "held back" by McDonnell and Seumas Milne, his communcations and strategiy director. Having fought for decades to win control of the party, the left is not prepared to relinquish it now. Few believe that McDonnell, a more abrasive character than the Labour leader, could win the 34 MP nominations required to stand (the 2007 and 2010 leadership contender also stated on 26 June that he would "never" run again). But some have floated the idea of guaranteeing him a place on the ballot in return for Corbyn standing down. 

***

Despite Corbyn’s landslide victory only 10 months ago, his opponents are hopeful of a close contest. They speak of an “unmistakable shift” in party members’ attitudes since the EU referendum defeat. Unpublished polling is said to show “significant public appetite” to sign up for the leadership election in support of an alternative to Corbyn.

But the Labour leader’s backers are unambiguously bullish about his chances. “The infrastructure with which we’re starting this campaign is so far ahead of where we were a year ago,” James Schneider, Momentum’s national organiser, told me. “And that was a stand-out campaign.” 

Corbyn’s allies see no evidence of a “significant” shift in opinion among members and activists. Although one senior figure conceded that “we will have lost some support”, he spoke of the potential to recruit thousands of new voters from the wider left – “the people Jeremy has stood with for thirty years”. More than 15,000 are said to have joined the party in the last week with the intention of defending his leadership. Polling by Newsnight found that 59 per cent of members and 45 of 50 constituency chairs continue to support Corbyn. Allies also cite the backing of 240 councillors and a 230,000-strong petition. Though the statement issued by 10 affiliated trade unions stopped short of guaranteeing support for Corbyn in a contest, most would likely side with him. 

The Corbynites, anticipating victory, are already threatening retribution. One told me that measures such as mandatory reselection of MPs and lower nomination thresholds would be brought forward. “They don’t want it but they’ll get it,” he warned. “They’ve opened Pandora’s box.”

But it is deselection by the voters at large, in an early general election instigated by a new Conservative prime minister, that many MPs now fear most of all. They warn that Labour could endure its worst result since 1935 - when it was reduced to just 154 seats. “There isn’t a safe seat north of Islington,” a senior figure quipped in reference to Corbyn's constituency. 

Never in its history has Labour endured such a profound rift between its members and its MPs. Some of the latter speak of a "unilateral declaration of independence" by the PLP if Corbyn is re-elected. This option, first advocated by Harold Wilson's former press secretary Joe Haines in the New Statesman, would involve the 172 rebels claiming the status of the official opposition by virtue of their size. As Haines wrote: "Remember, the PLP cannot be dictated to within the party by any outside body. If the MPs decide they want to elect their own leader of the PLP they can do so." 

The irony remains that without the support of MPs, Corbyn could never have become leader. It was the 36 nominations he achieved almost exactly a year ago on 15 June 2015 that enabled him to stand for the leadership. Fewer than half of those MPs wanted to see him elected. Labour's "old right" warned of the dangers of allowing him on the ballot in a election open to non-party members. The "border controls" imposed on the left in the 1980s, they noted, had been dismantled. But MPs persisted out of a desire for a "broad debate". They certainly got their wish. 

George Eaton is political editor of the New Statesman.

This article first appeared in the 30 June 2016 issue of the New Statesman, The Brexit lies