On November 20th Sony announced the established of Sony AI to advance fundamental research and development of AI to be applied to many fields. Their first focus will be on gaming, photography and … gastronomy. I’ll address gastronomy in the next post.
That gaming and photography are playing field for Artificial Intelligence shouldn’t come as surprise. In gaming you want to imbue smartness in the various artificial characters that interact with the player and you also want the game interface to understand in the most natural way the player(s).
In photography Artificial Intelligence has already being used as a tool in image recognition to help focussing on what matter most (what is catching our brain attention, usually the eye of a person), to activate trigger at the right moment (like waiting for people to smile and more recently to avoid to capture them as the eyelids are shut), to work out the correct exposure…
All of these “helps” are pretty difficult because they require the camera to recognise the overall image, detecting people, their eyes, lips, eyelids… as well as weighting their importance in the overall shot. There may be several people in a frame, which are the ones that matter most? People may not be smiling at all, but that might actually be the reason why the photographer wants the shot. A person may be sleeping and of course her eyelids are shut, and the photographer wants to take that shot, not a different one awaiting for that person to wake up. You see, the point is that image recognition is tricky and it can be pointless unless the camera can also understand the context. Now both recognition and context understanding require intelligence (some sort of it). Also, determining a correct exposure is difficult. This is why modern camera tend to have a technically correct exposure that is not representing what our eye sees! Try to take a photo in a dark alley with a digital camera and you get a photo where the dark alley is no longer “dark”. Sure, the camera digital sensor can see way better than our cones and rods but what we usually want is an image that is a faithful copy of what we perceive. AI can help.
We have seen progress in the last two years. Some digital cameras can now detect animals eyes (not jsut for dogs, like in the image, also for birds, reptiles, fish… and although our brain is prompt in recognising all different shapes that we call “eye” for a machine this is really complex, we have no problem in telling a leopard eye from its spots, yet they may look alike -but eyes are on its head, and in very specific places, something that we have no problem in “knowing”).
This “image recognition and understanding” capability is being used in post processing. The new Luminar 4 makes an amazing job in recognising where the sky is (no that is not simple! It may be hidden among the branches and leaves of a tree…) although it is far from perfect. As an example Luminar can replace a dull sky in a photo with a dramatic sunset but it is missing the fact that this new sky will be reflected on windows and on a lake surface. That goes beyond its -current- capability and so you have to take care of that. On the other hand Luminar lets you introduce effects like sun-rays and can work out pretty well the effect of those artificial sun rays on objects in the image (not perfect but, most of the time, quite good). Reflection is a hard issue to manage correctly because different surfaces reacts differently to light, reflecting it in different ways and creating subtly different shading and mirroring artefacts. The problem is that our brain uses those fine nuances to understand the world and any mismatch is promptly detected.
Adobe is launching in this days a free AI-powered smartphone app (to a selected group of photographers, I applied but I do not know if I will end up in that circle) where AI is delivered though their Sensei platform, and several other companies are advertising AI powered apps. As usual when looking at AI claims one should be cautious to their veracity and depth…
There is an additional aspect in photography that would require a higher level of AI: creativity. There have been example of AI being able to detect creativity in photos, AI has been applied to select the best photos in tournaments and amazingly AI choices are in synch with expert photographers choosing (that is because AI was trained to detect those traits, but nevertheless we have to agree that AI has been able to learn!).
Not sure if Sony will explore creativity in photography, and to what extent, but I tend to bet they will. IN their announcement headline they state the goal of “unleashing human creativity” so one might interpret it in various ways, including the flanking of AI creativity to the human one to this latter benefit.
Imagine having a drone and letting the drone taking beautiful photos deciding where and when… Amazing!