Our visual sense is steering our perceptions, our dogs apparently are more influenced by the olfactory sense, but nevertheless vision is just part of the story. Unless the brain receives inputs from the other senses that confirm what the eyes are telling it will detect that something is weird and the sensation of reality will get skewed. This is the case when we are watching a video at an IMAX theatre. Yes, you get the immersive sensation, you feel tricked in somehow falling along with the rollercoaster on the screen because your field of vision is completely embedded in the video but at the same time your proprioceptors are sending to your brain a discordant message, the one that your body is not moving at all. Hence the feeling of nausea that many people experience.
Even in situation where movement is not involved, the brain is quick to detect that something is wrong, sometimes just because it does not receive confirmation signals from the other senses.
Technology is progressing to re-create sensations involving all our senses. Visual and aural sensations can now be created at a fidelity level that matches the sensitivity of our senses, i.e. technology can trick the brain into believing it is getting real signals (4k video when filling our extended field of vision does that, hifi sound surround does the same to our ears).
Tricking our sense of smell, taste, touch and movement is more difficult. These four senses, however, are needed only in specific situations, if you are not in the process of eating (virtually) the taste does not come into the picture. Similarly, if you are in a virtual space where you are not moving the perception of movement is not required. Smells is often a complementary sensation, so our brain (differently from the one of our dog) does not raises a red flag if no smell sensation is coming. If you are immersed into a visual perception of a beach and waves breaking on the rocky shore. Visual and aural fidelity are essential to feel like being there. The salty smell that you would perceive if you were actually there would be a nice to have but if it is not there your brain is not going to scream “fake, fake…”.
The sense of touch, likewise, is not always needed but whenever there is a situation where in the real world the touch is part of the experience its lacking would immediately result in an artefact sensation, the brain would immediately become aware that something is wrong and it is not the real thing.
Providing tactile sensations that can be so accurate to fool our brain is quite difficult but progress is being made.
There are haptic devices that can recreate the exact sensation of using a tool, like in surgery training haptic scalpel have been used for several years providing the future surgeon brain the exact sensations she will experience when performing surgery on a patient. In flight simulators haptic controls are in widespread use, recreating the sensation of the real cloche. Actually, the digital stick on the fly-by-wire Airbus is haptic, it provides the pilot the feeling of being directly connected to the moving parts of the wing and rudder through a controlled force-feedback mechanism.
What about recreating the sensation of reaching out in a conference call and touching an object that is being shown to us from remote or touching the hand of our kids as we talk to them from thousands of miles away? Well, there are haptic gloves that can provide a -almost- accurate sensation but they are bulky and cannot provide the full spectrum of sensations. For sure they don’t feel like the real thing. Besides, when you touch something it is a two way communications when that something is a living thing. You touch your kid and both of view feel the touch. This is not possible, yet. If on your side there is a need to recreate the touch sensation over your hand (that is where a haptic glove comes in) on your kid side there would be a need for having actuators all over his body, or a robot that can impersonate you and your actions actually touching (with your touch) your kid. We are nowhere near making this happening.
On the possibility of having a haptic surface overlaid on the skin of your kid, so that wherever you touch the sensation can be evocated, we have a number of prototypes that indeed can recreate a sensation of being touched. The problem is that these prototypes are bulky, even when they are in form of a plastic fabric to place on the skin. You can create the sensation of being touched (limited to the area where the haptic skin is layered) but at the same time your kid brain will perceive that there is an object on the body skin and that will destroy the feeling of reality.
Using a robot, or a robotic arm to touch as a proxy present similar perception problems. Yes, the kid can feel your touch, via the robotic proxy, but will also perceive the robot, hence destroying the feeling of proximity.
May be, by the end of the next decade we will have an artificial skin that could be layered on our skin without being perceived as an external object and provide tactile sensations from far away. It remains to be seen. Personally, I do not see it coming in the next ten years.
What may happen is a scaled down version of a wearable haptic skin, and we are seeing the porno industry chasing this evolution to complement virtual reality goggles to make it … as real as it gets.
Just not to forget a “technical” issue, communicating a touch sensation requires establishing a feedback loop, you –the toucher- need to feel that you are touching and he –the touched- needs to feel the touch. These two are related and require very low latency in communications, in the order of a millisecond. As the latency increases you –both- start realising there is something funny going on, and once you have reached the hundred millisecond latency (delay) the sensation is no longer feeling real at all. Hence the communications infrastructure is playing a crucial role. 5G may provide a better support in this area but it is not enough.
This is, obviously, another area where the concept of “digital reality” may be opposed stressing that the more you provided a real-life experience the more you are messing up with reality. Again, a negative perception of Digital Reality.