Sound and vision: Kraftwerk perform in New York, 2012. (Photo: Getty)
Show Hide image

Having trouble with your vision? There’s an app for that

EyeMusic will allow you to hear shapes and colours

Having trouble with your vision? There’s an app for that. It’s called EyeMusic, and it’s the result of remarkable research at the Hebrew University of Jerusalem. It will allow you to hear shapes and colours, training your brain to process sensory information in a completely different way – and it’s rewriting what we thought we knew about how the brain works.

A report published in the 17 March edition of the journal Current Biology has surprised researchers by demonstrating that people who have been blind from birth can be trained to recognise silhouettes of the human body in various postures. The training involves around ten hours of listening to sounds. Once trained, the research subjects’ brains used sound – employing the same area as sighted people use – to recognise body postures.

This is surprising, because researchers have always assumed that the neurons used for visual processing will be reassigned in someone whose brain isn’t getting visual inputs. Received wisdom had it that they might, for instance, be used by the auditory system to enhance processing of sound signals. The idea is related to the frequent (but wrong) assertion that if you lose one sense, others become more sensitive. Studies suggest that the sense of smell is no more acute in blind people. Neither are they sharper of ear; they are perhaps marginally better at making use of auditory information, but they don’t actually hear better.

Not that making better use of sounds is to be sniffed at. There are visually impaired people who use bat-like echolocation to navigate their way through busy streets, for example. Perhaps most extraordinary is the Californian Daniel Kish. Thanks to an aggressive childhood cancer, Kish has no eyes. At the age of two he began to train himself to use the echoes of his tongue-clicks as a guide to his surroundings.

Kish can go out for a bike ride safely and hike alone through mountainous terrain. He has refined his skill to the point where his clicks and their echoes provide him with a 3D mental image of his environment. He teaches his craft to other blind people, liberating them from the severe restrictions that are so common for those without sight.

Amir Amedi’s lab in Jerusalem has a similar goal. With training, Amedi says, visual areas of the brain can process sound and touch just as successfully as they would visual inputs.

This month’s paper makes that case forcefully. Written
by Amedi’s student Ella Striem-Amit, it focuses on a region of the brain known as the extrastriate body area, which is devoted to recognising body shapes. In Striem-Amit’s adult, fully blind subjects, this area had had no training whatsoever. However, their extrastriate body area still functioned perfectly – it just needed a few hours of priming.

Amedi’s EyeMusic iPhone app runs the user through a library of sounds associated with visual patterns and shapes. Once the training is complete, sound combinations begin to suggest visual scenes.

It will take a while before you become as competent as the trainee who can play a sonic version of Guess Who? by associating sounds with facial characteristics such as hair colour, beardedness or spectacles. But it won’t take long to appreciate that your brain is a marvellously flexible organ that, with the right training, can give you the next best thing to superpowers. The app also collects performance data that will help Amedi develop his research.

Why play Candy Crush when you can idle away your time helping science give sight to the blind? 

Michael Brooks holds a PhD in quantum physics. He writes a weekly science column for the New Statesman, and his most recent book is At the Edge of Uncertainty: 11 Discoveries Taking Science by Surprise.

This article first appeared in the 03 April 2014 issue of the New Statesman, NEW COLD WAR

Getty/New Statesman
Show Hide image

Pupils need internet classes? Here are 41 lessons they should learn

Forget privacy and security, here's what to do when a black and blue dress looks white and gold. 

It is imperative that children are taught how to survive and thrive on the internet, claims a new House of Lords report. According to the Lords Communication Committee, pupils need to learn how to stay safe, avoid addictive games, and become “digitally literate”.

It’s hard to argue with the report, which is a great step forward in acknowledging that the internet now basically = life. Yet although it is crucial that children learn how to stay private and secure online, there are also some equally crucial and not-at-all-flippant pieces of information that the youth urgently need to know. Here are the first 41 lessons in that curriculum.

  1. To figure out how much to donate towards your mate’s charity half-marathon, half X OR double Y, where X is the amount paid by their mum and Y is the amount donated by your closest rival, Becky
  2. Don’t mention that it’s snowing
  3. If – for some reason – you talk about bombs in a Facebook message, follow this up with “Hi Theresa May” in case Theresa May is looking, and then Theresa May will think you are just joking
  4. If you are on a train and you are annoyed about the train, do not tweet @ the social media manager who runs the account for the train, because they are not, in fact, the train
  5. If a Facebook meme starts “Only 10 per cent of people can get this puzzle right” – know that lies are its captain
  6. It’s not pronounced me-me
  7. Never say me-me nor meem, for they should not be discussed out loud
  8. People can tell if you’ve watched their Instagram stories
  9. People can’t tell if you’ve waded back through their Zante 2008 album and viewed all 108 photos
  10. People can tell if you’ve waded back through their Zante 2008 album and viewed all 108 photos if you accidentally Like one – in this circumstance, burn yourself alive
  11. Jet fuel can melt steel beams
  12. If a dog-walking photo is taken in the woods and no one uploads it; did it even happen?
  13. Google it before you share it
  14. Know that Khloe Kardashian does not look that way because of a FitTea wrap
  15. Do not seek solace in #MondayMotivation – it is a desolate place
  16. Respect JK Rowling
  17. Please read an article before you comment about a point that the article specifically rebutted in great detail in order to prepare for such comments that alas, inevitably came
  18. Don’t be racist, ok?
  19. Never, under any circumstances, wade into the Facebook comment section under an article about Jeremy Corbyn
  20.  If a dress looks white and gold to some people and black and blue to some others, please just go outside
  21. Open 200 tabs until you are crippled with anxiety. Close none of the tabs
  22. Despite the fact it should make you cringe, “smol puppers” is the purest evolution of language. Respect that
  23. Take selfies, no matter what anyone says
  24. Watch Zoella ironically until the lines of irony blur and you realise that the 20 minutes you immerse yourself into her rose-gold life are the only minutes of peace in your agonising day but also, what’s wrong with her pug? I hope her pug is ok
  25. Nazi Furries are a thing. Avoid
  26. Use Facebook’s birthday reminder to remember that people exist and delete them from your Friends list
  27. When a person you deleted from your Friends list inexplicably comes up to you IRL and says “Why?” pretend that your little cousin Jeff got into your account
  28. Don’t let your little cousin Jeff into your account
  29. “Like” the fact your friend got engaged even if you don’t actually like the fact she is reminding you of the gradual ebbing away of your youth
  30. No one cares about your political opinion and if they act like they do then I regret to inform you, they want to have sex with you
  31. Please don’t leave a banterous comment on your local Nando’s Facebook page, for it is not 2009
  32. Accept that the viral Gods choose you, you do not choose them
  33. Joke about your mental health via a relatable meme that is actually an agonising scream into the void
  34. Share texts from your mum and mock them with internet strangers because even though she pushed you out of her vagina and gave up her entire life to help you thrive as a person, she can’t correctly use emojis
  35. Follow DJ Khaled
  36. Decide that “Best wishes” is too blah and “Sincerely” is too formal and instead sign off your important email with “Happy bonfire night”” even though that is not a thing people say
  37. If someone from primary school adds you as Friend in 15 years, accept them but never speak again
  38. The mute button is God’s greatest gift
  39. Do not tell me a clown will kill me after midnight if I don’t like your comment because that is not a promise you can keep
  40. Don’t steal photos of other people’s pets
  41. Accept that incorrect "your"s and "you’re"s are not going anywhere and save yourself the time 

Amelia Tait is a technology and digital culture writer at the New Statesman.