Home / Blog / Post processing before taking the shot!

Post processing before taking the shot!

An example of the post processing applied before the picture is taken showing the difference between the post processed image, on the right, with the image without post processing. Notice that the post processing is doing much more than just increasing the exposure time, in this example. It is also applying a correct white balance and specific exposure curves to highlight shadows still keeping the sky correctly exposed. Credit: MIT

Post processing lets you tweak your digital photos to improve the image (what it might mean to you: making it as similar as possible to the image you saw or enhancing some aspects to steer the image perception where you want) and that is why it is called “post processing”. You do it after you took the photo.

Now researchers at MIT have decided you could as well do your post processing before taking the shot, thus saving you time.

They developed a machine learning technology to tweak the image captured by the digital camera sensor in real time, presenting it on the screen of the smart phone you are using to take the picture. This lets you see the end result and of course you can tweak the processing to get the result you like.

Reality is usually a complex mix of shadows and lights, with different objects reflecting light in different ways (not just in terms of wavelengths -colour- but also in terms of quantity) hence each object would require a different exposure. This processing is done in our brain but a digital camera cannot do that, it just tries to average the overall image. With post processing you can finely tweak the exposure of individual objects and fine tune the white balance. The colour of the object depends on the temperature of the light illuminating it and this temperature is different at different time of the day, in different weather conditions and of course in artificial light. Again our brain is pretty good in doing this white balancing, and it does this applying “semantics”: at sunset the light is warmer -reddish- and our brain will not compensate for it because, well, it is sunset time. A digital camera, on the other hand, will detect the reddish lightning and will compensate generating an image that misses the “right” colours that we expect for a sunset scene.

These compensations and tweaking requires “intelligence” and “understanding” of what we, as end users, expect from a picture.  The team of researchers has trained their post processing algorithms to learn what we expect to see by having them learning on thousands of raw and retouched photos.

The overall process is quite time consuming and computation intensive, as anyone involved in post processing knows well. The challenge for the MIT team was to find a solution that could perform in real time using the computation power of a smart phone. And they succeeded (see clip).

What I find interesting is that the taking of a simple photo may involve such complex processing and that we are using artificial intelligence and machine learning to train the digital camera on how we perceive a picture and on what we consider a “good” picture.

We have come really a long way since the first photos taken on film. Now it is all about bits and how to manipulate them. A completely different game!

About Roberto Saracco

Roberto Saracco fell in love with technology and its implications long time ago. His background is in math and computer science. Until April 2017 he led the EIT Digital Italian Node and then was head of the Industrial Doctoral School of EIT Digital up to September 2018. Previously, up to December 2011 he was the Director of the Telecom Italia Future Centre in Venice, looking at the interplay of technology evolution, economics and society. At the turn of the century he led a World Bank-Infodev project to stimulate entrepreneurship in Latin America. He is a senior member of IEEE where he leads the New Initiative Committee and co-chairs the Digital Reality Initiative. He is a member of the IEEE in 2050 Ad Hoc Committee. He teaches a Master course on Technology Forecasting and Market impact at the University of Trento. He has published over 100 papers in journals and magazines and 14 books.