Clearly there is much more than voice based interaction. So let’s explore other technologies and their implication first by looking at voice (and typing that we have already considered) alternative technologies and then taking the plunge into multimodal interactions (that is our natural way of interacting as human being!).
Let’s start with touch. Normally we don’t use touch to communicate but there are several exceptions, like in dancing where the man steers the woman by using touch signals. This is particularly so with tango but it also goes with other dances. We use touch, sometimes, to communicate emotions, like the various ways in which you can touch a hand.
However, touch is an important sense in providing our brain with a sense of reality, actually it is so important that when it is missing we immediately perceive that something is fake, an artefact. This is evident when using virtual reality interfaces. These interfaces are not providing touch sensations and this is one of the main reasons why our brain feel VR as … virtual and not real. Sony has just hinted that the new Playstation 5, to hit the market at the end of 2020, will include an advanced haptic interface to make VR much more “real”.
Technologies for haptic interfaces have progressed in the last ten years. We are seeing them in several gaming devices (I have a force feedback joystick for a better flying simulator experience) and in professional equipment, like in robotic surgery control.
Haptic interfaces can be clustered in graspable, wearable and touchable.
Graspable interfaces are based on bars or sticks that change their resistance through a motor counteracting your forces (that is also why some of these are advertised as “force feedback” like my joystick). The evolution has been in accuracy, sensitivity and in number of directions (also known as degree of freedom). The very best graspable haptic can provide 7 degrees of freedom. To put this into perspective our human hand has … 27 degrees of freedom, hence we can experience much more subtle sensations.
Wearable haptics, like haptic gloves, are particularly useful in VR context since they can provide a touch sensation into thin air. The problem with these technologies is their bulkiness that by itself sends a message of “fake” (or artificial) to out brain. It is an area where significant progress has been made but where the cost is still high (probably not affordable to the mass market).
Touchable haptics, like the Apple 3D touch interface, are based on a vibrating surface. Depending on the vibration frequency it can recreate specific touch sensation. Apple introduced it back in 2015 in their iPhones but it is not present in their most modern ones, like XR, iPhone 11 and iPhone Pro (Apple claim to have replaced the 3d touch -that was a real haptic interface- with what they call a haptic interface -that is NOT a haptic interface since it just feels the length of time you keep your fingertips on the screen, it does not return any touch sensation…).
AR and VR would greatly benefit from seamless haptics and I am sure in the next decade further evolution in technology will bring this to our … fingertips. By the way, this is an area where very low latency is needed, hence where 5G could make a difference!