Home / Blog / Sleight of Ear

Sleight of Ear

This newspaper clip is totally fake. It has been generated by GPT-2 OpenAI using AI. The software has not been released – so far- because researchers felt it is too good and can spread misinformation like a wildfire. Image credit: OpenAI

Just in yesterday’s post I mentioned optical illusions and how we can trick our brain. Now I come up with an IEEE Spectrum article describing a machine learning based software, AutoFoley, developed at the University of Texas at San Antonio, able to create sounds mimicking real sounds to the point of fooling our ears (and brain) into believing it is the real thing.

Sound effects have been the bread and butter in movie making. Sound of rain, explosion, mewing cats and … you name it have become an essential part of any movie soundtrack adding a sense of reality to the images.

AutoFoley has been designed to create any type of ambient sound, feeding its learning algorithm with real sounds and then letting it create the sound clip of the desired length. You can watch the clip of a fireplace below and hear the sound of wood and flames crackling. They feel absolutely like the real thing, yet they have been artificially created by AutoFoley.

The researchers have tested the sounds generated  with human audiences and by far they were perceived as recordings of real sound. One problem still open is how to synchronise the generated sound with the video. If you have a fireplace there is no problem since the crackling is completely random and it can fit any burning fireplace, but if you have the clopping sound of a horse galloping then you need to have the sound synchronised with the image of hooves hitting the ground. Any difference in time will be promptly detected by our brain and raise a red flag. Something is weird!

I have no doubt that also this problem will be addressed in the near term, it is a matter of pulling together image recognition and sound (and sound meaning) generation.

The big problem that remains, and that is getting bigger and bigger, is the near impossibility to tell what is real from what is artificial. This goes across a huge, and growing, spectrum of our life. Forgery of the past turns out to be a child’s play if compared with today’s fake generated in the cyberspace. Add to this the fact that fakes can be generated using artificial intelligence and you see the problem.


About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.