It used to be the national television and radio programs and the providers new basically nothing about me (the state, in Italy, knew you had a television set because they charged you an annual “tax” on that).
Now, I subscribe to Netflix, to Apple TV, to Amazon Prime, and I use Youtube and a variety of apps to get programs from Rai (Italian broadcaster), Mediaset (a private Italian broadcaster) and many radio stations. I actually have access to more entertainment in a single day that I can ever hope to enjoy in a life time. And the rate at which the content expand on these entertainment sources is way beyond my capability to digest it. I receive quite a few emails every day soliciting my attention on this or that new content, made available “just for me”. And this is the starting point of this post. Although there are tons of content the providers are trying more and more to provide my special entertainment channel and they can do this because they know a lot about me!
The graphic I am showing is interesting because it present three basic classes of “my” data being shared with the entertainment providers. The graphic has been created by Netflix, possibly the largest entertainment company in terms of customers (close to 120 million) and coverage (190 countries) delivering some 140 million hours of entertainment each day. The discussion that follows should apply to most of the entertainment provider, although I based it on the data that Netflix is declaring to accrue.
First are the data I provide when I sign up. I am asked who I am, my birth data, where I live, my credit card number, the composition of my family, my contact (email address and sometimes my phone number) and from where I am signing on (location, type of device used, IP address, type of network I am using to connect).
Second are the data that I generate as I browse their catalogue, watch the trailers, what I select, how I watch it (seeing the whole movie or watching it in chinks…), what I seem to like (did I stop watching something and never went back to it?), the frequency of use, what devices I use, the IP address. If I access from a computer they will also get info from cookies, on cookies, form web beacons, ads identifier, web history (in practice it is like they highjack my computer….). All of this is made in a transparent and “informed” way. They have a web page indicating all what they do. I bet you haven’t really read it (nor did I) and just in case you want to look at it click here.
Third they create data about myself by using data analytics on my data and on all other customers data. As an example, they twitted last year that 53 of their customers watched A Christmas Prince every day for 18 days in a row (they clearly liked it!). That innocent tweet raised quite an outrage on social networks since it showed that they are tracking their customers. Now, quite frankly, I cannot understand the outrage. It should be clear to everybody that Netflix is tracing what and how we are using their content. Aren’t they able to present that movie from the point we last stopped watching it? Aren’t they suggesting content that fits pretty well with our tastes? How could all of this be possible if they were not tracking us?
However, this “outcry” is also an indicator that many of us are paying very little attention to the data we keep sharing in the cyberspace.
The correlation they make across different customers provides information on trends, on what people like in a specific area, or difference in tastes among people using a Pc or a Mac, among people watching content on a smartphone or on a television set.
With all this tracing and tracking can it get any worse?
Back in the last Century (that means 25 years ago!) the MPEG group standardised the MPEG7. At that time I was working in the same company of Leonardo Chiariglione, the MPEG convenor (and guru) and we spoke quite a bit on the possibilities offered by MPEG. In particular MPEG7 -Multimedia Content Description Interface- was designed with the idea that in the future spectators will be interacting with specific objects included in a video. What you see in technical terms is a bunch of pixels and you brain process them and distinguishes among a person with a glass, a table, a bottle of wine on the table. Well, the idea was to have a standard letting content provider to characterise / tag the various objects that our brain would later identify and associate to these object some characteristics. That would have allowed a content provider to change a single object in a scene leaving all the rest unchanged.
OK. Now imagine yourself sitting in your living room on the couch watching a movie on your big screen television seeing an actor pouring some whisky from a red label bottle. As you are watching the movie your 14 yo kid is also watching the same movie in his bedroom on his computer. Only that what your kid is seeing is an actor pouring Coke in tha glass from a can of Coca Cola. Like magic the content has been adapted to the viewer to send the message that having a shot of that whisky is cool -for the living room viewer- and -at the same time- that drinking Coke feels great.
To make this magic reality we need to have MPEG7, which we have, and a way to understand who is watching what. For this all what is needed is a video camera that is already available on the laptop, smartphone and tablet, and is becoming more common on television screens. These cameras provide the image of the viewer to an application that can understand what is going on in the viewing ambient:
- Who is viewing, male, female , approximative age
- the level of attention on what is being shown
- the mood of the viewer and how this mood is influenced by what is being shown
Now, I appreciate that this feels a bit intrusive, actually, quite a bit. One of the reasons why this scenario hasn’t turned into reality is the fact that real time rendering in a video stream is very computational intensive and we simply did not have that capacity. However, the situation is changing. Current televisions can upscale a video signal to and HD one and HD into a 4K and so on. This is done through AI and smart algorithms that are not expanding one pixel into 4, rather they are guessing the meaning of that pixel and find out what would make sense to display if there was a higher resolution capability. These are the same algorithms that will let you delete an object on a photo and see it replace by something that makes sense in the context of that photo. The result is far from being perfect but in most case is simply amazing!
In this decade I expect we will be seeing something along these lines, probably starting in the video game area, then extending to the Augmented and Virtual Reality and finally into classic entertainment.
Video has become a major communication channel, the pandemic has accelerated the transition to video communications. Tools like Zoom are already providing ways to replace the background with one of your choice… Tiny steps, all pointing towards a fading boundary between the real and the artificial so fuzzy actually that everything becomes, is perceived, as real.