Just in yesterday’s post I mentioned optical illusions and how we can trick our brain. Now I come up with an IEEE Spectrum article describing a machine learning based software, AutoFoley, developed at the University of Texas at San Antonio, able to create sounds mimicking real sounds to the point of fooling our ears (and brain) into believing it is the real thing.
Sound effects have been the bread and butter in movie making. Sound of rain, explosion, mewing cats and … you name it have become an essential part of any movie soundtrack adding a sense of reality to the images.
AutoFoley has been designed to create any type of ambient sound, feeding its learning algorithm with real sounds and then letting it create the sound clip of the desired length. You can watch the clip of a fireplace below and hear the sound of wood and flames crackling. They feel absolutely like the real thing, yet they have been artificially created by AutoFoley.
The researchers have tested the sounds generated with human audiences and by far they were perceived as recordings of real sound. One problem still open is how to synchronise the generated sound with the video. If you have a fireplace there is no problem since the crackling is completely random and it can fit any burning fireplace, but if you have the clopping sound of a horse galloping then you need to have the sound synchronised with the image of hooves hitting the ground. Any difference in time will be promptly detected by our brain and raise a red flag. Something is weird!
I have no doubt that also this problem will be addressed in the near term, it is a matter of pulling together image recognition and sound (and sound meaning) generation.
The big problem that remains, and that is getting bigger and bigger, is the near impossibility to tell what is real from what is artificial. This goes across a huge, and growing, spectrum of our life. Forgery of the past turns out to be a child’s play if compared with today’s fake generated in the cyberspace. Add to this the fact that fakes can be generated using artificial intelligence and you see the problem.