Support 100 years of independent journalism.

  1. Spotlight
7 June 2019

Spotlight Leader: The danger of the Black Box

It is already the case that engineers and researchers are unable to explain the ways in which the AI algorithms they've created reach the conclusions that they do.

By Spotlight

There are many examples of fiction for previous decades that, with hindsight, seems to have predicted the modern age, but few appear as prescient – and from such a distance in time – as EM Forster’s The Machine Stops, written in 1909. The story begins with a scene that many have argued presaged internet use by a century. A woman sits alone in a room, surrounded by technology. She can summon food or entertainment at the touch of a button, but she is bothered by constant notifications from an unseen community – “she knew several thousand people” – but she never sees anyone in person. In a line every internet user will relate to, she says: “Here I am in the dark, wasting my time”.

But Forster’s story may come to seem even more prescient in the years to come. We already live in an time in which most people are not able to explain more than a superficial amount about the technology by which they are surrounded every day. This is a new situation for humanity, and a confusing one, but it is still possible to learn how things work. As artificial intelligence becomes more sophisticated and more widespread, people will begin to use technology that no human is able to explain.

In fact, this is already happening. It is already the case that engineers and researchers are unable to explain the ways in which the AI algorithms they’ve created reach the conclusions that they do. There are engineers developing self-driving cars who are not able to get the information about why, in a given instant of a journey, the car decided to act in a certain way. Deep learning techniques applied to medical science appear to be very good at predicting the onset of some diseases, but because the software has written itself, researchers are not able to say how they work.

Few technologists worry about a robot uprising. But there is increasing unease that if the technologies that run our world become inexplicable, we may stumble into the situation Forster predicted: “No one confessed the Machine was out of hand. Year by year, it was served with increased efficiency and decreased intelligence. The better a man knew his own duties upon it, the less he understood the duties of his neighbour, and in all the world there is not one who understood the monster as a whole.”

To access the full Spotlight supplement, click here.

Sign up for The New Statesman’s newsletters Tick the boxes of the newsletters you would like to receive. Quick and essential guide to domestic and global politics from the New Statesman's politics team. The New Statesman’s global affairs newsletter, every Monday and Friday. The best of the New Statesman, delivered to your inbox every weekday morning. A weekly round-up of The New Statesman's climate, environment and sustainability content. A handy, three-minute glance at the week ahead in companies, markets, regulation and investment, landing in your inbox every Monday morning. Our weekly culture newsletter – from books and art to pop culture and memes – sent every Friday. A weekly round-up of some of the best articles featured in the most recent issue of the New Statesman, sent each Saturday. A weekly dig into the New Statesman’s archive of over 100 years of stellar and influential journalism, sent each Wednesday. Sign up to receive information regarding NS events, subscription offers & product updates.