Home / Blog / Seeing what you are looking at

Seeing what you are looking at

The image on the left was shown to a person. By analysing the EEG pattern a computer was able to reconstruct the image on the right. Credit: University of Toronto Scarborough

The research carried out at the University of Toronto Scarborough seems to come from a science fiction book. The researchers wanted to look inside a “brain” to see what the brain was seeing, i.e. the image the eyes of that person were transmitting to the brain.

The results have been published on an article on eNeuro reporting for the first time the ability to reconstruct an image seeing by a person’s eyes by looking at the EEG (ElectroEncephaloGraphy) wavy lines.

The researchers (see the clip) sat a person in front of a screen displaying images of faces. A computer was trained to match the electrical signals captured by the EEG with the various images. Then the computer was asked to reconstruct the images based on the electrical activities.

This is the first time that an EEG is used to look inside the brain to see what that person is looking at.

As you can see in the figure, the reconstruction is not precise but it is anyhow impressive.  Also notice that we are quite far from really seeing what a person is seeing by just looking at that person EEG. A long, and specific training is needed to let the computer associate specific signals to image characteristics. Besides, the brain will generate completely different signals if it were looking to a shoe instead of a face (it does not work by pixels like our digital camera, a brain works mostly on semantics, a face and a shoe have completely different meanings).

Additionally, notice that the training is specific to the person, that is once you have trained the computer on that person brain you cannot use the computer to look inside a different brain (and most likely it will not work after a month since in the meantime the brain will have changed its way of processing that image).

Still, it remains a quite amazing result!

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.