Using Augmented Reality Applications to Visualize 3D Radiology Images
Imagine using 3D technology to visualize planning for liver transplants and surgery. Thanks to research from Jesse Courtier, MD, associate professor of clinical pediatric radiology, that could soon become a reality. Dr. Courtier is currently working with Bencin Studios to develop an imaging application for Microsoft HoloLens, the first self-contained holographic computer, enabling users to engage with digital content and interact with holograms in the surrounding real world.
The software application, called “RadHA” or Radiology with Holographic Augmentation allows users display 3D medical images superimposed on a real-world background.
The video below, which features a clickable 3D holograph of a CT scan from a patient with a liver mass, demonstrates how the application works. With the technology, radiologists can visualize human anatomy in layers and distinct colors. In this case, for example, the full liver appears white, the arteries appear red, the hepatic veins appear blue, the portal vein appears orange and the tumor appears purple. By viewing the liver’s tumor from all angles, physicians will have a better understanding of the tumor’s location so they can make the best choices for treatment options.
3D applications are growing in popularity and usefulness in health care, and UCSF Imaging is proud to be leading the way in the field of radiology. The 3DHEALS 2017 Global Conference was held at the Mission Bay Conference Center at UCSF earlier this year with the intention of bringing technologists and health care providers together to create meaningful health care products using 3D printing. UCSF Radiology was represented at the event by Dr. Courtier; Yi Li, MD; Ziba Mansoori, MD; Vanitha Sankaranarayanan, and Derrick Gillan. Drs. Courtier, Li and Mansoori spoke about the ongoing work of the Quantitative Image Processing Center (QUIPC) and Sankaranarayan and Gillan showcased it at an exhibit table.
The use of 3D technologies in the department doesn’t stop there. Like Dr. Courtier and the QUIPC team, Judy Yee, MD, is a leading innovator in the realm of radiology and 3D applications. During her tenure as a researcher and clinician, she has developed virtual holography CT colonography (CTC), a technology that takes data from CT and MRI scans and transforms it into 3D holographic images using 3D glasses, a laser stylus, and a special display.
The idea for Dr. Courtier’s use of 3D holographic technology for liver masses blossomed while he was enrolled in the UCSF Entrepreneurship Program’s UCSF Startup 101 class in spring of 2017. He will work on developing the RadHA application further with assistance through the UCSF QB3 entrepreneurship support program.
Click here to learn more about how Dr. Courtier and QUIPC are bringing the latest image processing technologies to UCSF Imaging.