A French soldier maneuvers a remotely-controlled IED detecting robot during a training exercise at a combat outpost base in Usbeen village in Surobi district of Kabul province on March 13, 2012. Photo: Getty Images
Show Hide image

This week the UN is going to debate the ethics of killer robots

Machines that can choose who to kill independently of a human operator are coming, to the concern of ethicists and roboticists alike.

The United Nations is due to debate killer robots later this week. However, "killer robots" do not exist at this moment in time in the sense that’s relevant to the UN - that is, robots that can independently choose to kill humans. This isn’t about drones, even if drones are very good at killing - it’s more about what happens when the decision to fire a fatal shot, in any context, moves out of the hands of humans and into the circuitry of a computer.

The discussion, to be held during the UN Convention on Conventional Weapons (CCW) in Geneva from 13 May, will take the form of “an informal meeting of experts”, reports the BBC. Its conclusions will be delivered in a report to the CCW committee in November.

Here is how the BBC lays out the issue:

A killer robot is a fully autonomous weapon that can select and engage targets without any human intervention. They do not currently exist but advances in technology are bringing them closer to reality.

Those in favour of killer robots believe the current laws of war may be sufficient to address any problems that might emerge if they are ever deployed, arguing that a moratorium, not an outright ban, should be called if this is not the case.

However, those who oppose their use believe they are a threat to humanity and any autonomous 'kill functions' should be banned."

That opposition is represented by the Campaign to Stop Killer Robots - what a name for a campaign group! - which has already produced several of its own reports to argue against autonomous killing machines. The opening debate of the convention will feature the CSKR’s Noel Sharkey, a computer scientists from the University of Sheffield whom older readers may recall as a judge on Robot Wars(!); his opponent, from the Georgia Institute of Technology, is roboticist/roboethicist Ronald Arkin. (There’s a full itinerary for the convention available here.)

Why the worry, though? Because, on current trends, our ability to create autonomous machines will outpace our ability to program them. This is illustrated quite well by driverless cars, which seem to on track to enter real-world use sometime around the end of the decade.

A driverless car requires more than a mere ability to drive along paved roads, on a predetermined route, avoiding obstacles. There are myriad scenarios where a driverless car, like a human driver, will have to decide what to do in an emergency. That could mean killing itself. It could mean killing its own passengers.

Imagine driving down a high street and a child runs out into the road. A human might instinctively hit the brakes, trying to stop, even if the car isn’t physically able to do so. A computer, however, may well realise that a better course of action would be to steer a sharp turn around the kid, to narrowly avoid it. Or it may realise that clipping it at an angle will result in, say, a 40 percent chance of a serious injury, compared to 90 percent if hit straight on. (These statistics are hypothetical, but such things will be modelled by automobile manufacturers.)

Let’s say a driverless car is confronted with two choices: carry on straight, and plow into another vehicle with a family of four people in it; or turn off the road in either direction to avoid the other car, but in the process crash, possibly fatally for the passengers inside. What should it choose? What parameters should the car look to maximise? Total lives saved? Should it value two people, alive, but without the use of their legs, as better or worse than one person, alive, intact? What about three? Or four? Does it matter if the car is a Volvo or a convertible?

What if the car chooses to deliberately kill itself, and its own passenger, instead of risking the lives of those in the other car?

Philosophers have struggled for decades with issues like these - they’re known as trolley problems, after the 1967 paper by Philippa Foot which introduced the concept. Imagine watching a runaway train carriage heading down a hill, towards a group of five men working on the tracks - they’re too far away to shout a warning, and the carriage will undoubtedly kill them all. However, you’re next to a signal switch. Flick it, and the train is diverted into a siding with only one man at work. He will die, but the five others will live. Do you flick the switch?

Like all good thought experiments, the trolley problem is useful for showing us the gap between the material reality of an ethical quandary and our gut emotional response to it. In brute utilitarian terms, flicking the switch is obviously the right thing to do - but that doesn’t mean we’re comfortable with it, and that sensation of being uncomfortable gives us pause to think things through more th

Other formulations of the trolley problem - like killing a fat man and pushing him onto the tracks to stop the train and save the other five men - make it clear that there’s more nuance to this, and that there’s something that feels wrong about choosing to kill someone compared to letting them die.

However, a driverless car heading at 70mph down a motorway doesn’t have the luxury of ruminating on the hypothetical - it has real lives to consider, and it has to make decisions that are defined by human choice. This introduces strangeness to our idea of responsibility, and to guilt - and as Patrik Lin writes at Wired in his excellent feature on ethical autonomous cars:

Programming a car to collide with any particular kind of object over another seems an awful lot like a targeting algorithm, similar to those for military weapons systems. And this takes the robot-car industry down legally and morally dangerous paths."

Which brings us back to killer robots. Emma Woollacott wrote last week in the NS about the possible impossibility of teaching robots to understand ethics in the same way that humans do - that is, to feel emotional responses to ethical decisions, like feeling guilty for breaking a rule. In that absence, they have to be programmed to respond to situations like the trolley problem in a way that humans would accept.

As the debate at the UN will explore, maybe the only clean, acceptable way out of this entire debate - the most morally acceptable way out, if you will - would be to ban “Lethal Autonomous Weapons Systems” (as the UN calls them) altogether. Perhaps philosophers will find that they're suddenly employable, as arms and car manufacturers seek their advice out on acceptable moral frameworks to stick into new products. In science fiction, we can rely on Asimov's Three Laws of Robotics - if only we could do the same with our robots.

Ian Steadman is a staff science and technology writer at the New Statesman. He is on Twitter as @iansteadman.

Photo: Getty Images/AFP
Show Hide image

Is Yvette Cooper surging?

The bookmakers and Westminster are in a flurry. Is Yvette Cooper going to win after all? I'm not convinced. 

Is Yvette Cooper surging? The bookmakers have cut her odds, making her the second favourite after Jeremy Corbyn, and Westminster – and Labour more generally – is abuzz with chatter that it will be her, not Corbyn, who becomes leader on September 12. Are they right? A couple of thoughts:

I wouldn’t trust the bookmakers’ odds as far as I could throw them

When Jeremy Corbyn first entered the race his odds were at 100 to 1. When he secured the endorsement of Unite, Britain’s trade union, his odds were tied with Liz Kendall, who nobody – not even her closest allies – now believes will win the Labour leadership. When I first tipped the Islington North MP for the top job, his odds were still at 3 to 1.

Remember bookmakers aren’t trying to predict the future, they’re trying to turn a profit. (As are experienced betters – when Cooper’s odds were long, it was good sense to chuck some money on there, just to secure a win-win scenario. I wouldn’t be surprised if Burnham’s odds improve a bit as some people hedge for a surprise win for the shadow health secretary, too.)

I still don’t think that there is a plausible path to victory for Yvette Cooper

There is a lively debate playing out – much of it in on The Staggers – about which one of Cooper or Burnham is best-placed to stop Corbyn. Team Cooper say that their data shows that their candidate is the one to stop Corbyn. Team Burnham, unsurprisingly, say the reverse. But Team Kendall, the mayoral campaigns, and the Corbyn team also believe that it is Burnham, not Cooper, who can stop Corbyn.

They think that the shadow health secretary is a “bad bank”: full of second preferences for Corbyn. One senior Blairite, who loathes Burnham with a passion, told me that “only Andy can stop Corbyn, it’s as simple as that”.

I haven’t seen a complete breakdown of every CLP nomination – but I have seen around 40, and they support that argument. Luke Akehurst, a cheerleader for Cooper, published figures that support the “bad bank” theory as well.   Both YouGov polls show a larger pool of Corbyn second preferences among Burnham’s votes than Cooper’s.

But it doesn’t matter, because Andy Burnham can’t make the final round anyway

The “bad bank” row, while souring relations between Burnhamettes and Cooperinos even further, is interesting but academic.  Either Jeremy Corbyn will win outright or he will face Cooper in the final round. If Liz Kendall is eliminated, her second preferences will go to Cooper by an overwhelming margin.

Yes, large numbers of Kendall-supporting MPs are throwing their weight behind Burnham. But Kendall’s supporters are overwhelmingly giving their second preferences to Cooper regardless. My estimate, from both looking at CLP nominations and speaking to party members, is that around 80 to 90 per cent of Kendall’s second preferences will go to Cooper. Burnham’s gaffes – his “when it’s time” remark about Labour having a woman leader, that he appears to have a clapometer instead of a moral compass – have discredited him in him the eyes of many. While Burnham has shrunk, Cooper has grown. And for others, who can’t distinguish between Burnham and Cooper, they’d prefer to have “a crap woman rather than another crap man” in the words of one.

This holds even for Kendall backers who believe that Burnham is a bad bank. A repeated refrain from her supporters is that they simply couldn’t bring themselves to give Burnham their 2nd preference over Cooper. One senior insider, who has been telling his friends that they have to opt for Burnham over Cooper, told me that “faced with my own paper, I can’t vote for that man”.

Interventions from past leaders fall on deaf ears

A lot has happened to change the Labour party in recent years, but one often neglected aspect is this: the Labour right has lost two elections on the bounce. Yes, Ed Miliband may have rejected most of New Labour’s legacy and approach, but he was still a protégé of Gordon Brown and included figures like Rachel Reeves, Ed Balls and Jim Murphy in his shadow cabinet.  Yvette Cooper and Andy Burnham were senior figures during both defeats. And the same MPs who are now warning that Corbyn will doom the Labour Party to defeat were, just months ago, saying that Miliband was destined for Downing Street and only five years ago were saying that Gordon Brown was going to stay there.

Labour members don’t trust the press

A sizeable number of Labour party activists believe that the media is against them and will always have it in for them. They are not listening to articles about Jeremy Corbyn’s past associations or reading analyses of why Labour lost. Those big, gamechanging moments in the last month? Didn’t change anything.

100,000 people didn’t join the Labour party on deadline day to vote against Jeremy Corbyn

On the last day of registration, so many people tried to register to vote in the Labour leadership election that they broke the website. They weren’t doing so on the off-chance that the day after, Yvette Cooper would deliver the speech of her life. Yes, some of those sign-ups were duplicates, and 3,000 of them have been “purged”.  That still leaves an overwhelmingly large number of sign-ups who are going to go for Corbyn.

It doesn’t look as if anyone is turning off Corbyn

Yes, Sky News’ self-selecting poll is not representative of anything other than enthusiasm. But, equally, if Yvette Cooper is really going to beat Jeremy Corbyn, surely, surely, she wouldn’t be in third place behind Liz Kendall according to Sky’s post-debate poll. Surely she wouldn’t have been the winner according to just 6.1 per cent of viewers against Corbyn’s 80.7 per cent. 

Stephen Bush is editor of the Staggers, the New Statesman’s political blog.