Support 100 years of independent journalism.

  1. Spotlight
7 June 2019

Spotlight Leader: The danger of the Black Box

It is already the case that engineers and researchers are unable to explain the ways in which the AI algorithms they've created reach the conclusions that they do.

By Spotlight

There are many examples of fiction for previous decades that, with hindsight, seems to have predicted the modern age, but few appear as prescient – and from such a distance in time – as EM Forster’s The Machine Stops, written in 1909. The story begins with a scene that many have argued presaged internet use by a century. A woman sits alone in a room, surrounded by technology. She can summon food or entertainment at the touch of a button, but she is bothered by constant notifications from an unseen community – “she knew several thousand people” – but she never sees anyone in person. In a line every internet user will relate to, she says: “Here I am in the dark, wasting my time”.

But Forster’s story may come to seem even more prescient in the years to come. We already live in an time in which most people are not able to explain more than a superficial amount about the technology by which they are surrounded every day. This is a new situation for humanity, and a confusing one, but it is still possible to learn how things work. As artificial intelligence becomes more sophisticated and more widespread, people will begin to use technology that no human is able to explain.

In fact, this is already happening. It is already the case that engineers and researchers are unable to explain the ways in which the AI algorithms they’ve created reach the conclusions that they do. There are engineers developing self-driving cars who are not able to get the information about why, in a given instant of a journey, the car decided to act in a certain way. Deep learning techniques applied to medical science appear to be very good at predicting the onset of some diseases, but because the software has written itself, researchers are not able to say how they work.

Few technologists worry about a robot uprising. But there is increasing unease that if the technologies that run our world become inexplicable, we may stumble into the situation Forster predicted: “No one confessed the Machine was out of hand. Year by year, it was served with increased efficiency and decreased intelligence. The better a man knew his own duties upon it, the less he understood the duties of his neighbour, and in all the world there is not one who understood the monster as a whole.”

To access the full Spotlight supplement, click here.

Select and enter your email address Quick and essential guide to domestic and global politics from the New Statesman's politics team. A weekly newsletter helping you fit together the pieces of the global economic slowdown. The New Statesman’s global affairs newsletter, every Monday and Friday. The New Statesman’s weekly environment email on the politics, business and culture of the climate and nature crises - in your inbox every Thursday. Our weekly culture newsletter – from books and art to pop culture and memes – sent every Friday. A weekly round-up of some of the best articles featured in the most recent issue of the New Statesman, sent each Saturday. A newsletter showcasing the finest writing from the ideas section and the NS archive, covering political ideas, philosophy, criticism and intellectual history - sent every Wednesday. Sign up to receive information regarding NS events, subscription offers & product updates.
  • Administration / Office
  • Arts and Culture
  • Board Member
  • Business / Corporate Services
  • Client / Customer Services
  • Communications
  • Construction, Works, Engineering
  • Education, Curriculum and Teaching
  • Environment, Conservation and NRM
  • Facility / Grounds Management and Maintenance
  • Finance Management
  • Health - Medical and Nursing Management
  • HR, Training and Organisational Development
  • Information and Communications Technology
  • Information Services, Statistics, Records, Archives
  • Infrastructure Management - Transport, Utilities
  • Legal Officers and Practitioners
  • Librarians and Library Management
  • Management
  • Marketing
  • OH&S, Risk Management
  • Operations Management
  • Planning, Policy, Strategy
  • Printing, Design, Publishing, Web
  • Projects, Programs and Advisors
  • Property, Assets and Fleet Management
  • Public Relations and Media
  • Purchasing and Procurement
  • Quality Management
  • Science and Technical Research and Development
  • Security and Law Enforcement
  • Service Delivery
  • Sport and Recreation
  • Travel, Accommodation, Tourism
  • Wellbeing, Community / Social Services
Visit our privacy Policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications.

Topics in this article :