Surgeon Shafi Ahmed is pictured wearing a Microsoft HoloLens headset inside his operating room at the Royal London Hospital on Thursday, January 11, 2018.
Bloomberg | Bloomberg | beautiful pictures
Metaverse, the Next Big Thing of the digital world, is billed as the internet domain where animated avatars of our real selves will be able to perform virtually all kinds of interactive activities, from shopping to gaming to travel – someday. Wonks says it could take a decade or more before the necessary technologies catch up to the hype.
Right now, however, the healthcare industry is using several essential ingredients that will eventually include surreal – virtual reality (VR), augmented reality (AR), mixed reality (MR) and augmented reality (AR). ) and artificial intelligence (AI) – as well as the software and hardware to power their applications. For example, medical device companies are using MR to assemble surgical instruments and design operating rooms, the World Health Organization (WHO) is using AR and smartphones to train medical professionals. COVID-19 responders, psychiatrists are using VR to treat post-traumatic stress (PTS) among combat soldiers, and medical schools are using VR for surgical training.
Facebook, Oculus and Covid
Since Facebook – now Meta . Platform – acquired Oculus and its VR headset technology in 2014 for $2 billion, many healthcare applications have been developed. One of the latest events is a partnership with Facebook Reality Labs and Nexus Studios and WHO Academy. The organization’s R&D incubator has designed a mobile learning app for healthcare workers battling Covid-19 around the world. One of the trainings involved AR to simulate on a smartphone the proper techniques and sequences for putting on and taking off personal protective equipment. With content available in seven languages, the app builds on the needs of 22,000 global health workers surveyed by WHO last year.
Oculus technology is used at UConn Health, the University of Connecticut medical center in Farmington, Connecticut, to train residents in orthopedic surgery. Educators have partnered with PrecisionOS, a Canadian medical software company that offers VR training and orthopedic education modules. Wearing the Oculus Quest headset, residents can visualize in 3-D performing a series of surgical procedures, such as placing a pin in a broken bone. Because the process is virtual, the system allows students to make mistakes and get feedback from the instructor to include in the next attempt.
Meanwhile, as the metaverse is still being built, “we see a great opportunity to continue the work that Meta has been doing in support of medical efforts,” a Meta spokesperson said. “As Meta’s experiences, apps, and services evolve, you can expect the wellness strategy to play a role, but it’s too early to say how that might intersect with other technologies and vendors. like third-party level.”
When Microsoft introduced its HoloLens AR smart glasses in 2016 for commercial development, including Stryker, a medical technology company in Kalamazoo, Michigan. In 2017, they began harnessing AR equipment to improve the design process of operating rooms for hospitals and surgical centers. Because the OR is shared by different surgical services — from general surgery to orthopedic, cardiac, and others — surgical lighting, equipment, and instruments vary depending on the procedure.
Realizing the opportunity HoloLens 2 presents in evolving 2D to 3D OR designs, Stryker engineers are able to design shared ORs using three-dimensional images. The MR experience visualizes all people, devices, and setups without requiring the presence of objects or people.
Zimmer Biomet, a medical device company based in Warsaw, Indiana, recently unveiled its OptiVu Mixed Reality Solutions platform, using HoloLens devices and three apps – one using MR for manufacturing surgical tools, another collects and stores data to track a patient’s progress before and after surgery, and a third allows clinicians to share MR experiences with patient before the procedure.
A spokesperson for Zimmer Biomet said: “We are currently using HoloLens in a pilot fashion with remote support in the US, EMEA and Australia. The technology is already being used for training and case coverage programs remotely, and the company is developing software applications on HoloLens as part of its centralized data solutions, the spokesperson said. before and after the procedure, the spokesperson said.
Microsoft’s three-dimensional vision of the future
In March, Microsoft introduced Mesh, an MR platform powered by the Azure cloud service, that allows people in different physical locations to experience 3-D holograms on other devices. each other, including HoloLens 2, a range of VR headsets, smartphones, tablets and PCs. . In a blog post, the company imagined avatars of medical students, learning about human anatomy, centered around a hologram and dissecting the back muscles to see what’s underneath. .
Microsoft sees many opportunities for its MR technology and in March secured it $20 billion contract with US military for use with soldiers.
In real-life applications of medical AR technology, Johns Hopkins neurosurgeons performed the organization’s first AR surgery on living patients in June. In the initial process , doctors placed six screws into the patient’s spine during spinal fusion. Two days later, a separate team of surgeons removed a cancerous tumor from one patient’s spine. Both teams are equipped with headsets made by Augmedics, an Israeli company, which are equipped with see-through screens that project images of the patient’s internal anatomy, such as bones and other tissues, based on CT scan. “It’s like having a GPS navigator in front of your eyes,” says Timothy Witham, director of the Johns Hopkins Neurosurgery Spine Coordination Laboratory.
At the University of Miami’s Miller School of Medicine, faculty members at the Gordon Center for Simulation and Innovation in Medical Education use AR, VR, and MR to train first responders to treat trauma patients, including those who have had a stroke, heart attack or gunshot wound. Students practice life-saving heart procedures on Harvey, a lifelike dummy that simulates practically any heart condition. Wearing a VR headset, students can “see” basic anatomy graphically exposed on Harvey.
“In the digital environment, we are not bound by physical objects,” says Barry Issenberg, MD, Professor of Medicine and Director of the Gordon Center. Before developing a virtual technology curriculum, he said, students must be in the field and train on real-life trauma patients. “We can now ensure that all learners have the same virtual experience, regardless of their geographic location.”
Since its founding in 1999, the University of Southern California’s Institute of Innovative Technology (ICT) has developed VR, AI, and other technologies to address a variety of medical and mental health conditions. “When I first got involved, the technology was the Stone Age,” recalls his tinkering process, said Albert “Skip” Rizzo, psychologist and director of medical virtual reality at ICT. with an Apple IIe and a Game Boy handheld game console. Today he uses Oculus VR and AR headsets, HP and Magic Leap.
Rizzo helped create a VR exposure therapy, called Bravemind, that is aimed at PTS relief, especially veterans of the wars in Iraq and Afghanistan. During exposure therapy, a patient, guided by a trained therapist, confronts their traumatic memories through simulations of their experience. Wearing the headset, patients can immerse themselves in a number of different virtual scenarios, including a Middle Eastern themed city and desert road environment.
“Patients use the keyboard to simulate people, insurgents, explosions, even smells and vibrations,” says Rizzo. And instead of just relying on imagining a specific scenario, patients can experience it in a secure, virtual world as an alternative to traditional talk therapy. Evidence-based Bravemind therapy is currently available at more than a dozen Veterans Administration hospitals, where it has been shown to significantly reduce PTS symptoms. Additional randomized controlled studies are underway.
As Big Tech continues to build the metaverse, along with software and hardware companies, academia, and other R&D partners, the healthcare industry remains a real-life proof-of-concept. Paulo Pinheiro, head of software at Sagentia Innovation, based in Cambridge, UK, writes on the consulting firm’s website: “Although the metaverse is still in its infancy, it has enormous potential for transform and improve healthcare. “It will be interesting to see how the situation plays out.”
