Prostock-Studio/iStock
In a world that is dominated by terms like metaverse and virtual reality (VR), the term mixed reality (MR) might seem like one that harbors less potential. However, this could not be further from the truth since MR can also be thought of as the next step in the evolution of VR. This explainer will help you understand the complexities of MR and why you haven't heard of it as much.
The term VR has been thrown around much lately thanks, in part, to the years of the pandemic where some yearned for more innovative and immersive ways to interact remotely, which was followed by Mark Zuckerberg's public pivot to building a future of the internet in this whole new dimension.
Many companies that were working in this space seized this opportunity to unveil their plans for the metaverse and Web 3.0. However, as time passed and life went back to the pre-pandemic normal, the consumer interest in VR appears to be waning. When it comes to MR, though, user interest has just begun to pique.
Virtual reality is an environment where one can interact with digital objects in a fully immersive space. Here, the user experiences the environment using a specially designed headset that creates a virtual world in which they interact, irrespective of the real-world conditions around them.
In contrast, augmented reality (AR) overlays digital objects over the real-world environment in our surroundings in a semi-immersive manner where the user can see the objects but is also aware of their surroundings.
Mixed reality is a blend of both AR and VR and puts together the best of both worlds, creating digital objects that one can interact with over the real-world environment that the user is in. Of course, like VR, AR and MR also require an intermediary device – such as interactive glasses or a smartphone – to display the digital layer.
Since the MR experience is not completely immersive, it can be created on a variety of displays ranging from computer monitors to graphic displays. The overlays of reality and virtual environments can be varied on such displays and can include real video displays and digital objects at one end to fully graphic environments and real physical objects on the other.
Depending on the immersive requirements of these environments, display environments for MR range from monitors to handheld devices and heads-up displays to head-mounted ones. As our engineering capabilities improve, display devices are expected to get smaller in the future, starting with eyewear such as glasses and shrinking to contact lenses and virtual retina displays.
Progress made in this field so far has largely been able to produce headset displays at commercial levels, which is where we shall focus our attention this time.
Mixed reality headsets are wearable computing devices that combine the different aspects of the VR and AR worlds to deliver the mixed reality experience. Depending on the kind of MR experience required, the headsets sit somewhere between the wide range spectrum of VR and AR.
As discussed earlier, MR displays could be tasked with providing a digital overlay of an app or video onto the real world, giving the individual an option to interact with the objects therein without removing the user from the real-world surroundings. To do so, a wide range of technologies need to work together.
A host of sensors and cameras on the headset or device need to keep a track of the user's location and movements. Using data from these inputs, the device can then map objects in the user's surroundings to create realistic experiences.
In order to create believable experiences, the MR device may deploy either stereoscopic or holographic displays. Stereoscopic displays work much like a 3D movie and use two separate images to create the illusion of depth. A combination of the images creates a 3D effect.
Holographic displays, on the other hand, use light projection which creates digital objects that appear to float in midair. While holographic displays are more realistic, they are harder to produce, and most headsets end up using stereoscopic images.
Mixed reality might appear largely about visuals. However, the right mix of sounds and haptics can make the experience more convincing. Currently, there are two ways; this is being attempted with sound, using either binaural audio or spatial audio.
Like stereoscopic vision, binaural audio uses two separate streams of sound to create a realistic soundscape. This is a tried and tested technology and is cheaper to deploy. However, at times, the experience created is a bit unnatural, and therefore spatial audio is also used.
Spatial audio is a relatively newer technology and uses algorithms to create a more authentic sense of distance and direction, making the experience more realistic and natural.
A technology still in its early days, the application of mixed reality is still being trialed out in various fields. Other than delivering interactive content for purposes of entertainment, there are many other areas where technology could be useful.
The first among them is learning, whether at a school or college level, or in a professional setting. Learning can be greatly enhanced if the recipient is fully immersed in the topic being studied. MR could also allow the users to gain hands-on experience, either through virtual objects or content delivery. From repairing vehicles to performing surgeries, MR can help in training across the industry spectrum.
In areas such as construction and engineering, MR headsets can be used to visualize products as well as projects. Discussions about possible changes to the design and its impact on the product can be visualized instantly helping make faster decisions and smoother executions.
Inspections of production lines whether industrial or agricultural can be performed without traveling to the site using mixed reality, while video calls and meetings which became a norm in the pandemic years, could get even more interactive and realistic with this technology.
With so many technologies to pack inside a headset, the final device is not at all pocket friendly for the average retail consumer. Microsoft, which has a working product called the Holo Lens in this domain has a hefty price tag of $3,500, although this product is designed for commercial use, where the high price tag is not as limiting. It is possible that Apple's much-rumored upcoming headset, which is rumored to carry a similar price tag, might also offer some features that could put it into the MR category.
Retail VR headsets that are now available from scores of providers are relatively cheaper to purchase but also come with limited functionality. The added advantages of MR command a higher price but might also push it beyond the reach of tech enthusiasts for now. Instead, the applications developed for these headsets could be largely industrial or military, at least until the costs come down a few years from now.
If the leap to VR looks too big to make right now, very soon, it will become the stepping stone to a brand new world of digital realities. The rapid development of associated technologies, such as 5G internet and new-age processors, can help decrease latency and make the experience seamless.
Lighter-weight materials and battery technology will make headsets of the future easier to use, while AI-powered learning will help MR experiences more realistic in the future. As businesses look to get more people onboard, innovations will push for lower barriers to entry in this space, making it as common as smartphones today.
Such a future may not be very far from now.
IE talked with Shohini Ghose about how quantum computers might transform our future, the mysteries of quantum mechanics, and what the quantum scene will look like in 2027.
