Although consumers are becoming very familiar with speech recognition, current discussions of the metaverse and extended reality (XR) omit, for the most part, adequate speech and audio can be played in virtual or augmented environments. After all, most humans interact almost constantly with their environment in making and identifying sounds.
This is a major omission. In fact, as the drive for a compelling metaverse experience continues, the goal of true immersion will only be achieved when virtual landscapes or elements can provide the combination of all the sensations that allow users to fit in with these environments.
Siri has become one of the most popular speech assistants since Apple introduced the feature in 2011. The technology dates back to a SRI International project from 2003 The research institute was established as an independent company in 2007. This company was acquired by Apple in 2010 and since then, Amazon.com has introduced Alexa and Google has created its own Google Assistant.
Microsoft has its own speech recognition feature and in March 2022 it got it Nuance CommunicationsProvider of speech recognition and artificial intelligence (AI) technologies. Coincidentally, Nuance is partly owned by another subsidiary of SRI International acquired in 2005. before becoming part of Microsoft.
The sound can become the story
But sound can be more than that, and movie and game developers have shown that sound can become the story – whether it’s the rattling of a clock or a faint, almost echoing heartbeat, not to mention explosions and the like.
Flexund It has developed technology that can be embedded in theater or gaming chairs, as well as cushions or even furniture. The system enables the creation of personal sound fields and allows users to feel the sound with their bodies. while, Sennheiser has developed immersive sound Which Netflix now uses in some of its movies and select scenes. Sennheiser has also created AMBEO, which provides spatial audio for a wide range of applications, including Cinematic virtual reality.
Also, many companies already offer services for designing meaningful audio environments for niche applications—even if their current focus isn’t on XR or metaverse-related applications. These efforts range from fairly comprehensive services to well-targeted applications.
Makani CompanyFor example, it develops soundtracks for retail outlets, hospitality environments, office spaces, and museums, among other locations. Cine Sound, meanwhile, focuses on designing more pleasant and less stressful sounds for hospitals and healthcare sites. It’s easy to see how this understanding of design will find its way into creating more feasible and immersive augmented reality (AR) and virtual reality (VR) applications.
The crisp touch
Another set of interface technologies already playing a role in high-end games will help XR apps become more immersive and, in many cases, more comprehensive and convenient. touch It can enable use cases that are currently difficult to translate from real to virtual environments.
Touches are divided into tactile and kinesthetic sensations. Tactile experiences mainly relate to the skin, such as the perception of texture, touch, pressure or vibration. Kinesthetic experiences relate to muscles, tendons, joints, weight perception, and stretching, but also to the movement of body parts. Different interfaces already exist and will evolve to address these kinds of sensibilities.
The sense of touch in the hands is perhaps the most helpful. Holding an instrument, pressing something, or feeling a surface in virtual environments will improve the authenticity of the experience if haptic feedback is added. At the most basic level, haptic feedback will enable virtual presentations of standard interfaces, such as buttons that can be pressed or dials that spin and provide some sense of clicking.
A number of companies offer gloves as interfaces. HaptX It features both types of feedback and includes motion tracking for applications in virtual reality or for remote robotics operations. SenseGlove It offers an advanced solution that enables users to learn about the size, density and resistivity of objects in virtual reality. The sensations in the fingers and hands of manipulating and exploring objects in VR provide clear benefits.
So it’s no surprise that Facebook’s Reality Lab is as well Try the tactile glove to understand its capabilities. Other companies are trying to achieve similar sensations without having to wear a glove. ultraleap He is one of these and uses ultrasound to show tactile sensations on the hands.
These different approaches will be used in different environments and for different use cases. Ultrasound can be used more easily in public settings and for augmented reality, where the need to wear gloves may create friction in the experiment. Meanwhile, physical gloves can create more varied sensations and represent certain objects more accurately.
To create a full sense of immersion, full suits can deliver enveloping sensations. To become fully one with virtual environments for entertainment, training, diagnostic and therapeutic applications, systems that can provide tactile sensations to the upper body or the whole body are welcome additions. One of those bHaptics ‘ A line of so-called TactSuit products, which are essentially vests that incorporate haptic feedback spread all over the body. The company also offers a haptic glove.
tlassoIn the meantime, it offers a line of tangible clothing: the Teslaglove and the Teslasuit, which consists of a jacket and pants. Here the tactile sensations are based on electrical stimulation.
More immersive virtual environments
Researchers are also experimenting with different techniques for creating tactile sensations. A group of scientists at the University of Chicago said they “identified… Five chemicals can make tactile sensations permanent: numbness (sanshool), anesthesia (lidocaine), stinging (cinnamaldehyde), warming (capsaicin) and cooling (menthol).” These tactile devices include a sheath that surrounds part of the forearm and a tape that can be Placed below the visible headset on the user’s cheeks.
Perhaps attachable accessories for mass-market headphones—similar to the way University of Chicago researchers use their tape in conjunction with headphones—will become a niche in the market to provide users with certain sensations. Feelrealwhich started as a Kickstarter campaign, is a shield-like device that attaches to the bottom of a visual headset. device can Provides sensation From cool breezes to warm splashes of water and a wide variety of scents.
while, OVR technology It introduces a device that can be connected to a VR headset to provide a wide range of scents. The company mentions use cases such as meditation and response training for this device. Leveraging VR users’ sense of smell should come naturally to developers seeking to create more immersive virtual environments. Aaron Wisniewski, CEO of OVR, says:It would be a metaverse without smell like living life in black and white. “
These extensions can provide a wide range of interaction possibilities. A research team at Salzburg University of Applied Sciences has developed AirRes, an add-on for the Meta Quest 2 headset. The device looks like a gas mask, which the researchers describe as having a breathing interface. It enhances the wearer’s breathing as input information via an enable resistance valve Interact with virtual wind machinesOr inflate a birthday cake or use a blowpipe to propel projectiles, for example.
The resistance device can also restrict the user’s breathing, thus simulating entering a smoke-filled room, for example.
The applications in games or emergency training are obvious. Some of the applications presented by the researchers point to the device’s ability to improve virtual environments with new levels of reliability. A slight sigh in the mirrors of their haze can reveal hidden numbers left behind, similar to the way children leave messages for each other in mystery games.
Fogging of glass or metal surfaces can create a sense of realistic interaction with objects within environments and offers the possibility to leave messages for other players on virtual windows or metal surfaces for gaming purposes, for example.
while, H2L Technologies He developed a wrist strap that could deliver pain via small electric shocks. “Feeling pain allows us to turn the crosshairs into a real world, with Increased feelings of presence and immersion. Although the device can provide a sensation of pain, its primary purpose is to create a sense of resistance and weight when this occurs. Interact with objects in virtual reality.
Finally, brain interfaces could be the ultimate connection to virtual worlds and augmented landscapes. Such connections to XR environments are currently speculative, but with time and the development of neuroscience, fairly simple applications will become possible, and companies are already exploring related technologies.
Meta Platforms’ Reality Labs is researching the use of a brain-computer interface (BCI) for augmented reality glasses, especially for Use of BCI in communication applications. Others also see benefits in combining augmented reality glasses with brain interfaces. In March 2022, pop Acquired NextMind, a developer of BCIs. Snap is presumably exploring using such an interface with its smart AR glasses, Snap Spectacles. Possible uses for the interface could include games or playback devices, for example.
Meanwhile, Elon Musk’s Neuralink is generally developing interfaces that allow “A direct link between the brain and everyday technologyIn 2021, the company released information on macaques The ability to move cursors on the screen The video game Pong is played by brain activity alone.
Currently, many of the developments in the XR and metaverse are experimental, and interface technologies are part of these experiments. Truly immersive environments will require advanced interfaces that can replicate real representations in the real world. But unwieldy and difficult-to-use interfaces can prevent users from immersing themselves in VR as well.
Costs are another consideration, and safety will play an increasingly important role. But VR applications will drive the search for interfaces that enable more immersive interactions with virtual objects and worlds — and advanced interfaces will enable more complex XR applications.
Martin Schwern is the author of Small Data, Big Disruption: How to Spot Signals of Change and Manage Uncertainty (ISBN 9781632651921). He is also a senior advisor and strategic visionary at Business Finland, helping start-ups and established companies find their place in the market of tomorrow.