Augmented reality beats MRI for identifying liver tumors

Norwegian researchers have demonstrated how they accurately identified liver tumors more quickly by examining 3D models with augmented reality (AR) technology, compared with looking at conventional MRI scans.

Standard practice for the preoperative planning of liver tumor resection involves examining patient anatomy on MRI or CT scans -- a time-consuming method that offers a limited spatial understanding of tumors and surrounding anatomical structures, noted first author Dr. Egidijus Pelanis and colleagues from Oslo University Hospital in Norway. As a result, various groups have begun exploring the possibility of using 3D models to facilitate the identification of key anatomical structures.

To that end, Pelanis and colleagues acquired the MRI scans of a healthy volunteer's liver and used multiple imaging software applications (ITK-Snap; 3D Slicer) to segment the liver parenchyma and blood vessels into nine distinct liver segments. Then they superimposed different sets of artificially generated tumors onto the segmented MRI data (Minimally Invasive Therapy & Allied Technologies, 22 May 2019).

Next, the group converted the 2D images into virtual 3D models that were compatible for viewing using a proprietary software application installed in an AR headset (HoloLens, Microsoft). While wearing the headset, users were able not only to visualize the 3D models but also to interact with them through hand gestures.

3D liver model visualized through AR technology. Image courtesy of Dr. Egidijus Pelanis.

The researchers invited attendees at the 2018 Young Surgeons of the Hungarian Surgical Association annual meeting to identify liver tumors by examining either the AR models or MRI scans of each of the nine segments. In all, 28 physicians with a median of six years of medical experience participated.

Overall, the physicians were able to determine the precise location of liver tumors just as accurately with the AR models as on the MRI scans -- but nearly four times faster. The only segment for which they noted a statistically significant difference in the accuracy of tumor detection was segment 7, which greatly favored AR over MRI (87.5% vs. 25% detection rate).

The ability of clinicians to locate liver tumors more quickly on AR models than on MRI scans suggests that using AR could help speed up the learning curve for the task, the authors noted. Whereas conventional MR or CT images require clinicians to mentally reconstruct a 3D view of the liver vessels and lesions, AR eliminates the need for this step.

"The time for this 2D-into-3D transformation, together with clinical assessment, can be decreased almost four times using 3D models from segmented liver images in combination with [the AR] head-mounted display," they wrote. "The time saved outside and inside the operating room may lead to a significant decrease in operating time, which would result in improved use of resources and an economic benefit."

On a follow-up questionnaire, most of the physicians gave the AR headset a high rating for comfort, and 92% of the physicians affirmed they would recommend the AR device to their colleagues. The feedback indicates that clinicians are interested in applying AR to clinical practice, though additional validation studies are needed to assess the technology's clinical benefits, the authors concluded.