UBC researchers use Microsoft HoloLens to study the brain

Researchers at the University of British Columbia have created an app for the Microsoft HoloLens that allows students to use the mixed reality technology to study the brain.

“The brain is a very complex structure, in particular in its three dimensional anatomy,” says Claudia Krebs, professor of anatomy in UBC’s Faculty of Medicine, in a press release. “We think of it as a unit, but really, there are a bunch of sub-nuclei and different structures that work together. That makes neuroanatomy particularly difficult to teach.”

Enter the HoloLens, Microsoft’s mixed reality headset that allows the user to not just see computer-generated images as holograms within real space but to physically manipulate virtual objects, in this case, by being able to spin around a virtual brain floating in front of you, to dismantle it and then put it back together.

Brain power: “The first time I put on the HoloLens, I was blown away by what I saw…”

“The first time I put on the HoloLens, I was blown away by what I saw,” says Parker Holman, PhD candidate in neuroscience at UBC. “To be able to walk around and fully explore a detailed hologram of the brain from every angle is an experience that you can’t quite put into words.”

The new app, known as the Holographic Brain Project, is a product of a collaboration between UBC researchers, who first received ten HoloLens devices last year from Microsoft, and interns working for Microsoft’s project development arm, Microsoft Garage. Students in neuroanatomy classes this fall will be the first to try out the new tech at UBC.

“UBC is a growing innovation hub and we’re eager to explore how new partnerships and technology can help us continue to advance and transform our education and research space and methods,” says Gail Murphy, vice-president, research and innovation at UBC.

Using Microsoft HoloLens to study the brain…

Developed by Alex Kipman, creator of Microsoft Kinect, HoloLens is powered to calibrate the dimensions of a room and its contents by the users’ visual sweep (or gaze), allowing for digital content to become fully immersed in a user’s surroundings. In contrast to augmented reality experiences which superimpose computer graphics on top of a real-world environment (Pokemon Go is a common AR product), HoloLens is advertised as mixed reality, where the digital and real elements are said to be woven together in a more seamless fashion.

HoloLens applications have been developed for a variety of purposes, from game platforms and educational products to 3D modelling in architecture and virtual tourism. There are potential medical applications, as well, with one platform promising to use 3D tracking with the HoloLens to help train physicians to perform spinal surgery.

Earlier this month, Microsoft HoloLens won the Best in Show award at the 2017 International Design Excellence Awards.

In its latest news, Microsoft has reportedly filed a patent for a wand-like interface said to be intended for the HoloLens as a gaming ad-on. The HoloLens already comes with a clicker but the new device is promised as being adaptable for more precise actions like painting and first-person shooter games.