Support 100 years of independent journalism.

  1. Science & Tech
18 August 2016updated 30 Jun 2021 8:10am

What is merged reality and how will it affect your future?

Two new examples of “merged reality” technology have been making waves, but what does it actually mean?

By Amelia Tait

This is your future:


At least according to Intel and Microsoft, who, at the Intel Developer Forum in San Francisco on Tuesday, discussed two new developments. The first is Project Alloy, a wireless headset that allows you to bring real objects into the virtual world using 3D cameras. The second is Windows Holographic Shell, and it does the reverse – allowing virtual computer screens, objects, and people into the real world.

Both are examples of merged (or mixed) reality. Nice. Very good. Article over. Except, well, what does that mean?

What is mixed/merged reality?

In short, mixed reality is when the real and virtual world are merged and objects from both can interact. So, for example, if you were playing a virtual video game, you could pick up your nearby real-life water bottle and hit Dr Eggman over the head with it.

Specifically, Intel’s Project Alloy uses mixed reality to allow people to use their real hands in the virtual world. Before this point, users had to hold controllers that represented their hands. Now, the headset’s RealSense cameras will detect your finger movements and allow you to manipulate virtual objects. It is quite an exciting development, but it also looks like this:

Select and enter your email address Quick and essential guide to domestic and global politics from the New Statesman's politics team. The New Statesman’s global affairs newsletter, every Monday and Friday. Your new guide to the best writing on ideas, politics, books and culture each weekend - from the New Statesman. A weekly newsletter helping you fit together the pieces of the global economic slowdown. A newsletter showcasing the finest writing from the ideas section, covering political ideas, philosophy, criticism and intellectual history - sent every Wednesday. The New Statesman’s weekly environment email on the politics, business and culture of the climate and nature crises - in your inbox every Thursday. Sign up to receive information regarding NS events, subscription offers & product updates.
  • Administration / Office
  • Arts and Culture
  • Board Member
  • Business / Corporate Services
  • Client / Customer Services
  • Communications
  • Construction, Works, Engineering
  • Education, Curriculum and Teaching
  • Environment, Conservation and NRM
  • Facility / Grounds Management and Maintenance
  • Finance Management
  • Health - Medical and Nursing Management
  • HR, Training and Organisational Development
  • Information and Communications Technology
  • Information Services, Statistics, Records, Archives
  • Infrastructure Management - Transport, Utilities
  • Legal Officers and Practitioners
  • Librarians and Library Management
  • Management
  • Marketing
  • OH&S, Risk Management
  • Operations Management
  • Planning, Policy, Strategy
  • Printing, Design, Publishing, Web
  • Projects, Programs and Advisors
  • Property, Assets and Fleet Management
  • Public Relations and Media
  • Purchasing and Procurement
  • Quality Management
  • Science and Technical Research and Development
  • Security and Law Enforcement
  • Service Delivery
  • Sport and Recreation
  • Travel, Accommodation, Tourism
  • Wellbeing, Community / Social Services
Visit our privacy Policy for more information about our services, how New Statesman Media Group may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications.

The headset will also allow you to see your friends in the virtual world, and keep the confines of the virtual world in the real world space around you. In short, this means less walking-into-walls than in a traditional virtual world. 

The other development, Windows Holographic, uses the HoloLens headset to let you interact with holographic screens that display things like your emails and calendars. The HoloLens is also one single operating system for virtual reality, mixed reality, and augmented reality. According to Windows’ promotional video, you can also visit the Pantheon.

Windows via YouTube

How is it different from virtual and augmented reality?

Virtual reality is the umbrella term for all these magical artificial worlds that are popping up around us. Most commonly, users plonk on a headset and get a 360° view of an immersive fake world. Naturally, it’s thus far predominantly been used for video games and porn.

Augmented reality – of which the most famous example is Pokemon Go – is when virtual stuff is added to your real world environment. The virtual reality element, ie. Pikachu, is overlayed onto the real world. This has been used for sporting events and tourism, as information about the game or place can be overlayed as you look around. Although this is similar to merged reality, Microsoft and Intel’s new projects go further as they do not require sensors and are wireless, so more seemlessly merge the real and virtual worlds. 

When is mixed reality coming to destroy life as we know it forever?

Windows Holographic was unveiled last year, but the latest announcement revealed that it will be soaring into any Windows 10 user’s life next year. Project Alloy’s consumer release date is set for 2018, and for the time being, there is no word on the price. Although these are big developments, it could take years for the technology to become commonplace. In practice, although it sounds cool, there might not be much demand or it may be considered socially unacceptable (remember Google Glass, anyone?).

What will it change about your life?

Theoretically in a few years, you could attend a sporting event or concert without leaving your home. Ok, sure, TV has let you do this for a fair few decades, but Intel’s CEO Brian Krzanich explains the difference in a Medium post:

“Walk onto the field and right into the goalie box, or join the huddle, looking over the quarterback’s shoulder to see the calls before they’re played. Stand on the stage with your favorite band and feel the love of the fans from their point of view. Choose your experience and navigate it. It’s all up to you. The virtual world doesn’t need to be virtual; it can be real.”

Krzanich also theorises that you could practise tennis without visiting a court, go on a virtual holiday, and – for some reason, if you were so inclined  play the piano and cello at the same time.

Intel’s announcement trailer also features a man enjoying a virtual party under the premise: What if virtual reality felt less virtual and more real? The idea is that people could come together to party while physically being in different parts of the world, and that you could physically walk around the party without the need for cords and controls. Sounds great, right? Except they then demonstrate that you can pause the party and throw a waiter in a pool. The fact the comment section is disabled says more about this then we ever could:

For gamers, mixed reality will also potentially mean a better experience, whereby users can be completely immersed in fictional words by using their own hands and body instead of controllers. 

In the world of work, Windows Holographic could let you have meetings with people where you’re physically remote but virtually in the same place. 

Finally, according to Windows’ video demonstration, you’ll also have a fake puppy.

Windows via YouTube

No word yet on whether it leaves fake mess to clean up too.