Haptic suits, vests, and gloves for XR are known to have protective casings for exoskeleton designs that can add weight, wiring and a bit of extra heat to the wearer’s body. In Potsdam, Germany researchers Pedro Lopes, Sijing You, Alexandra Ion and Patrick Baudisch from the Hasso-Plattner-Institute at The Human Computer Interaction Lab, designed an Electrical Muscle Stimulation (EMS) system that can be used with the augmented reality (AR) of HoloLens and the mixed reality (MR) of Magic Leap One.
EMS, AR, and MR
Their innovative use of clinical style EMS pads and thin wiring connects to a backpack that sends electrical signals down the arm, forearm, and hands, which gets rid of the bulk of another piece of clothing or shelled wearable. The pulses aren’t meant to be overwhelming, they’re light enough to provide what’s called force-feedback, or the simulated feeling of weight and presence of a preconstructed image or by using a real-life object onto the user’s hands or fingers with AR or MR headsets.
The researchers explained the benefits of using their system, stating: “The benefits of our system, which have been validated in our user study, are: (1) providing force feedback to MR provides users leads to a better understanding of the virtual object’s state. (2) This haptic information is especially useful because the current headsets have a limited field of view. (3) The addition of force feedback to MR games and experiences increases the perceived realism.”
When the muscle stimulation and programmed headsets get paired together, it allows wearers to put the Hololens and Magic Leap One controllers aside for a hands-free and gestural approach to the researcher’s specially designed augmented and mixed reality games, experiences, and simulated control systems. Of these experiences, there is one that has the wearer use a prop like a lamp by pressing an AR button, another that turns a constructed dial to change an image, and there’s even an escape room to get out of.
See how the HassoPlattnerInstitute HCI EMS system for MR and AR works in practice below.
Real Potential
The researchers also tested their tech with a flat tray that has an AR image on it that renders images into the HoloLens headset. Participants held the tray, looked at the puzzle game that appeared, and then moved and manipulated the ball to the other side. This tipping of the tray and simulated weight and movement of the ball is important due to the fine motions and false weight and presence of the ball.
It’s important to see AR and MR’s potential within the fitness industry with these advancements. This study gives a real-life example of what can be done with real and imagined objects in a virtual space. The possibilities for game-related creations is widened for AR and MR industries with the EMS and force-feedback system. Imagine wearing a thin shirt with the tech built into it and then putting on AR or MR glasses at home or at the gym as you play more advanced games, especially hands-free.
Being hands-free and experiencing presence and weight of a sofa can potentially lead to other simulated objects with more weight and the ability to be moved or manipulated. Maybe there will be a huge Sisyphus-sized boulder to push across a room in Crossfit one day. If the researchers go the route of creating experiences around picking up and interacting with real objects maybe this could bring the fitness and health industries a step closer to pairing the tech with gym and physical therapy equipment.
Considerations
In their current state, the wiring and EMS pads are a bit clinical and less user-friendly for non-researchers. However, the tech could mean lighter haptic and force-feedback clothing in the future. Companies like Sourcenity and VisionBody have already successfully created a VR compatible EMS PowerSuit and an accompanying rhythm game by the name of PowerBeatsVR.
Live Strong reports that muscle stimulation can aid in muscle recovery and healing in therapeutic settings, but using the EMS tech for hours isn’t something MR and AR fanatics will want to do to get fit and healthy. Stimulating the muscles with electric pulses isn’t a realistic long-term solution to give you a shredded body; but being active, lifting weights and exercising is.
If researchers and companies pair up and guide each other towards building a better AR and MR wearable that’s also sweat and water resistant, they could apply it to fitness, training, and therapeutic settings. This could lead to more applications in gaming and experiences built around fitness, sports, physical therapy, and even entertainment.
How To Participate
MS Power User (MSPU) reports that the researchers have sites for open source code creation for HoloLens and an open source interactive EMS kit on GitHub. If you code and develop you can start helping them build up their system. Road to VR’s Scott Hayden reports that Pedro Lopes, Sijing You, Alexandra Ion and Patrick Baudisch will be presenting their research at the ACM CHI Conference in Montréal this year.