Home / Blog / Turning 2D into 3D through AR in surgery

Turning 2D into 3D through AR in surgery

What a doctor sees during surgery with AR goggles and MediView software. Image credit: MediView

OK, let me start apologising for the title. A bit too cryptic, but it captures the essence of this news.

Surgeons have now available plenty of images. most of them rendering of data analyses by a computer. These images are displayed on screens (even film images like radiography are displayed on flat panels and more an more on video displays), i.e. they are 2D representation of 3D objects, the body, organs and tissues of the patient.

The surgeon brain has to do the conversion from 2D to 3D to guide the surgeon’s hands into performing the right movements. This is tricky. Thinks about removing a tumour. The goal is to remove all cancerous cells and none of the normal cells. This is clearly impossible so the goal becomes to remove all cancerous cells and as few as possible of the normal cells. To stay on the safe side the surgeons is usually removing normal cells around the tumour to make sure that all cancerous cells are removed. The images are taken from different angles so that the surgeon’s brain can reconstruct the exact size and shape of the tumour combining the different 2D images into an actionable 3D shape.

This is not easy and this is where the software, and system, developed by MediView comes in to help. MediView is a start-up based in Ohio, US, developing tools for Extended Reality Surgical Navigation.

Using MS HoloLens or other devices supporting Augmented Reality the MediView software let the surgeon to see where the instrument she is using goes under the skin, into an organ, making use of all data harvested through exams and rendered in such a way to create a meaningful image overlaid on the physical patient.

The software can also be used (watch the clip) for training and to prepare for the surgery, visually recreating the operating field and the location shape and dimension of the tumour that will have to be removed. The surgeon can practice and experiment with different approaches “seeing” the anatomical structures in detail and at any level of magnification.

MediView started trials on 5 real patient in August 2019 and it is working to fine tune the software aiming at getting approval for clinical use by FDA in 2021.

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the Industry Advisory Board within the Future Directions Committee and co-chairs the Digital Reality Initiative. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.