The console camera of the future could track your movements through walls

MIT researchers have developed a camera that can build 3D models of users throughout a house - a potential game controller of the future.

The newest generation of consoles is going to get us used to being constantly watched by cameras in our living rooms. This might not be how Microsoft would put it, but that’s what the effect of an always-on Kinect will be. People have been penalised for swearing, at themselves, in their own living rooms:

To clarify, that’s a man getting an in-game penalty in NBA 2k14 because he swore outside of the game. Other games have been found to have similar features. Players can toggle the game’s listening-in on or off, but the important point here is that it’s incredibly creepy, and it’s the default, by choice of the developers. The console offers that function, so games will use it.

Let’s look forward, then, to what we might see in the generation after this one. We’re probably going to have cameras that track us through walls. Researchers at the MIT Computer Science and Artificial Intelligence Lab (CSAIL) have developed a system called “WiTrack” that uses radio waves to build 3D images of game players as they move throughout their home, even when they’re in other rooms.

It builds on an earlier system called “WiVi”, developed by some of the same researchers, that was a rough system for tracking people using the Wi-Fi on a typical smartphone. It works based on quite a simple, but clever, insight - the radio waves that a Wi-Fi device emits, like a smartphone or router, are deflected and reflected by objects they come across in just the same way that the radio waves used in radar systems do.

The resolution of Wi-Vi was pretty unimpressive - it could pretty much only tell you if something was moving closer towards you or further away - but it was an exciting idea. At the time, it was suggested that it could be used in hostage situations to figure out where people were inside buildings, or used by rescue teams to find those trapped beneath rubble in the aftermath of an earthquake.

WiTrack, conversely, uses its own radio waves instead of Wi-Fi, meaning it can build up full 3D models of its immediate environment. Here’s more info from MIT:

WiTrack operates by tracking specialized radio signals reflected off a person’s body to pinpoint location and movement. The system uses multiple antennas: one for transmitting signals and three for receiving. The system then builds a geometric model of the user’s location by transmitting signals between the antennas and using the reflections off a person’s body to estimate the distance between the antennas and the user. WiTrack is able to locate motion with significantly increased accuracy, as opposed to tracking devices that rely on wireless signals, according to Adib.

“Because of the limited bandwidth, you cannot get very high location accuracy using WiFi signals,” Adib says. “WiTrack transmits a very low-power radio signal, 100 times smaller than WiFi and 1,000 times smaller than what your cell phone can transmit. But the signal is structured in a particular way to measure the time from when the signal was transmitted until the reflections come back. WiTrack has a geometric model that maps reflection delays to the exact location of the person. The model can also eliminate reflections off walls and furniture to allow us to focus on tracking human motion.”

If that's not enough, there's a video that walks through how it works:

Gamers can look forward, in theory, to a future Kinect-like system with WiTrack capability that can track them throughout their homes. You could lead a squad of soldiers into your kitchen, make a sandwich, and head back to the action in the living room. Or, more usefully, it could be deployed in care homes to detect when elderly patients fall.

It is also, as a bit of technology that can literally see through walls, quite creepy. Imagine the privacy headache that could arise if everyone in a block of flats could see through into their neighbours’ homes, and imagine the paranoia that people would have that the NSA was keeping watch on them (especially considering that we now know that governments have infiltrated games like World of Warcraft).

It's still hard to see quite how this technology could be rolled out in a consumer setting, but give it time and there are probably hints of what's yet to come here.

A researcher using the WiTrack system. (Image: MIT)

Ian Steadman is a staff science and technology writer at the New Statesman. He is on Twitter as @iansteadman.

Screenshot of Black Mirror's Fifteen Million Merits.
Show Hide image

How likely are the plots of each Black Mirror episode to happen?

As the third series is on its way, how realistic is each instalment so far of the techno-dystopian drama? We rate the plausibility of every episode.

What if horses could vote? What if wars were fought using Snapchat? What if eggs were cyber?

Just some of the questions that presumably won’t be answered in the new series of Charlie Brooker’s dystopian anthology series Black Mirror, somewhere between The Twilight Zone with an app and The Thick Of It on acid.

A typical instalment takes an aspect of modern technology, politics, or life in general and pushes it a few steps into the future – but just how plausible has each episode been so far?

Series 1 (2011)

Episode 1: The National Anthem

Premise: A member of the Royal Family is kidnapped and will only be released unharmed if the Prime Minister agrees to have sexual intercourse with a pig on live television.

Instead of predicting the future, Black Mirror’s first episode unwittingly managed to foreshadow an allegation about the past: Charlie Brooker says at the time he was unaware of the story surrounding David Cameron and a pig-based activity that occurred at Oxford university. But there’s absolutely no evidence that the Cameron story is true, and real political kidnappings tend to have rather more prosaic goals. On the other hand, it’s hard to say that something akin to the events portrayed could NEVER happen.

Plausibility rating: 2 out of 5

Episode 2: Fifteen Million Merits

Premise: Sometime in the future, most of the population is forced to earn money by pedalling bikes to generate electricity, while constantly surrounded by unskippable adverts. The only hope of escape is winning an X-Factor-style game show.

In 2012, a Brazilian prison announced an innovative method of combating overcrowding. Prisoners were given the option to spend some of their time on electricity-producing bikes; for every 16 hours they spent on the bike, a day would be knocked off their sentence.

The first step to bicycle-dystopia? Probably not. The amount of electricity a human body can produce through pedalling (or any other way, for that matter) is pretty negligible, especially when you take account of the cost of the food you’d have to eat to have enough energy to pedal all day. Maybe the bike thing is a sort of metaphor. Who can say?

Plausibility rating: 0 out of 5

Episode 3: The Entire History of You

Premise: Everyone has a device implanted in their heads that records everything that happens to them and allows them to replay those recordings at will.

Google Glasses with a built-in camera didn’t work out, because no one wanted to walk around looking like a creepy berk. But the less visibly creepy version is coming; Samsung patented “smart” contact lenses with a built-in camera earlier this year.

And there are already social networks and even specialised apps that are packaging up slices of our online past and yelling them at us regardless of whether we even want them: Four years ago you took this video of a duck! Remember when you became Facebook friends with that guy from your old work who got fired for stealing paper? Look at this photo of the very last time you experienced true happiness!

Plausibility rating: 5 out of 5

Series 2 (2013)

Episode 1: Be Right Back

Premise: A new service is created that enables an artificial “resurrection” of the dead via their social media posts and email. You can even connect it to a robot, which you can then kiss.

Last year, Eugenia Kuyda, an AI entrepreneur, was grieving for her best friend and hit upon the idea of feeding his old text messages into one of her company’s neural network-based chat bots, so that she and others could, in a way, continue to talk to him. Reaction to this was, unsurprisingly, mixed – this very episode was cited by those who were disturbed by the tribute. Even the robot bit might not be that far off, if that bloke who made the creepy Scarlett Johansson android has anything to say about it.

Plausibility rating: 4 out of 5

Episode 2: White Bear

Premise: A combination of mind-wiping technology and an elaborately staged series of fake events are used to punish criminals by repeatedly giving them an experience that will make them feel like their own victims did.

There is some evidence that it could be possible to selectively erase memories using a combination of drugs and other therapies, but would this ever be used as part of a bizarre criminal punishment? Well, this kind of “fit the crime” penalty is not totally unheard of – judges in America have been to known to force slum landlords to live in their own rental properties, for example. But, as presented here, it seems a bit elaborate and expensive to work at any kind of scale.

Plausibility rating: 1 out of 5

Episode 3: The Waldo Moment

Premise: A cartoon bear stands as an MP.

This just couldn’t happen, without major and deeply unlikely changes to UK election law. Possibly the closest literal parallel in the UK was when Hartlepool FC’s mascot H'Angus the Monkey stood for, and was elected, mayor – although the bloke inside, Stuart Drummond, ran under his own name and immediately disassociated himself from the H’Angus brand to become a serious and fairly popular mayor.

There are no other parallels with grotesque politicians who may as well be cartoon characters getting close to high political office. None.

Plausibility rating: 0 out of 5

Christmas special (2015)

Episode: White Christmas

Premise 1: Everyone has a device implanted in their eyes that gives them constant internet access. One application of this is to secretly get live dating/pick-up artistry advice.

As with “The Entire History of You”, there’s nothing particularly unfeasible about the underlying technology here. There’s already an app called Relationup that offers live chat with “relationship advisers” who can help you get through a date; another called Jyst claims to have solved the problem by allowing users to get romantic advice from a community of anonymous users. Or you could, you know, just smile and ask them about themselves.

Plausibility rating: 4 out of 5

Premise 2: Human personalities can be copied into electronic devices. These copies then have their spirits crushed and are forced to become the ultimate personalised version of Siri, running your life to your exact tastes.

The Blue Brain Project research group last year announced they’d modelled a small bit of rat brain as a stepping stone to a full simulation of the human brain, so, we’re getting there.

But even if it is theoretically possible, using an entire human personality to make sure your toast is always the right shade of brown seems like overkill. What about the risk of leaving your life in the hands of a severely traumatised version of yourself? What if that bathwater at “just the right” temperature turns out to be scalding hot because the digital you didn’t crack in quite the right way?

Plausibility rating: 1 out of 5

Premise 3: There’s a real-life equivalent of a social media block: once blocked, you can’t see or hear the person who has blocked you. This can also be used as a criminal punishment and people classed as sex offenders are automatically blocked by everyone.

Again, the technology involved is not outrageous. But even if you have not worried about the direct effect of such a powerful form of social isolation on the mental health of criminals, letting them wander around freely in this state is likely to have fairly unfortunate consequences, sooner or later. It’s almost as if it’s just a powerful image to end a TV drama on, rather than a feasible policy suggestion.

Plausibility rating: 2 out of 5

Series 3 of Black Mirror is out on Friday 21 October on Netflix.