IEEE Future Directions
Related Articles
Posted on behalf of Louis Nisiotis and Lyuba Alboul
As a result of the recent outbreak of COVID-19 and the wide-reaching recommendations for discouraging or even prohibiting large gatherings, many conferences, concerts, matches and tournaments, and various meetings have either being postponed, cancelled or changed venues, with some of them deciding to go online. For instance, the prestigious IEEEVR and other scholar communities have opted for hosting their conferences in digital forms. Some universities and schools have suspended in-person classes and instead are moving to virtual instruction in order to contribute towards containing the virus spread and its associated health and safety issues. Microsoft has decided to offer 6 months of free access to their Teams collaborative online platform to support remote working. Even entire countries are on lockdown!
Hosting events in digital forms is not a new concept. Especially digital conferences organised by various scholar communities with expertise and innovative approaches to remote participation have been taking place with great success over the past few years. For example, the Virtual Worlds Best Practices in Education (VWBPE) conference, OpenSimulator Community Conference, and more recently the Educators in VR International Summit (behind the scenes), are examples of success stories where remote participation have been facilitated through the use of immersive technologies. Such digital conferences provide the opportunity to their participants to co-exist in the same virtual space, present their work, communicate and socialise through the comforts and safety of their own homes, their workplaces, or even when being on the go. This is achieved using a range of technologies such as through standard web browsers, specific virtual world viewers, web conferencing tools, virtual reality, and smart phones! Taking into consideration that within the next few years, 5G technology should also be delivered appropriately and fully implemented, emerging technologies are beginning to be at a state which their fusion can provide excellent and innovative results.
Eventually, the outbreak of coronavirus would be over sometime in the near future hopefully, and by then, a large number and wide range of audience will be introduced into remote participation in events through digital worlds, and this will no longer be an enthusiast’s or experimental activity. We cannot predict if this hype of digital attendance will continue, but what we know is that these kinds of activities offer opportunities for bringing people together from dispersed geographical locations. However, we also know that the main drawback of the “all-digital” conference experience approach is the lack of human to human interaction that cannot be replicated through the digital cyber space, and this is something that hinders the experience. Nevertheless, considering the recent advancements in immersive, robotic and smart technologies, could this be the time where the fusion of computer graphics and cutting edge robotic technologies can be employed to provide a hybrid approach and support conferences and/or other events?
A solution to this could be the convergence of Mixed Reality, Robots, and Social Networking technologies in ways that seamlessly blend the real and virtual worlds in a new type of amalgamated reality, in the form of a new type of conceptually led Cyber-Physical-Social Eco-Society systems (CPSeS). In such a system, the users can immerse and coexist in the same virtual space, interact with the physical and digital worlds in multimodal ways, and their behaviour will influence the real and the virtual environments in real-time. Such CPSeS would provide the opportunity for a hybrid approach into attending and participating in events and interact with people and objects, where travelling is difficult, restricted or even impossible due to financial, political and other reasons.
An example of such system is under development by researchers at Sheffield Hallam University. The research team is currently working on this system to support Cultural Heritage [1] [2] [3] [4], Arts and Exhibition, and Warehousing, and they are also aspiring to apply and support a plethora of domains. The developed system goes beyond current trends in AR and VR experience, by merging the physical with the virtual worlds through a fusion of cutting edge robotic technology for actuation purposes, immersive technologies for visualisation, and integration of social input between users in the real and virtual worlds. This is a multi-user mixed reality environment that comprises robots (Figure 1) and their corresponding virtual avatars (Figure 2) that link the physical with digital worlds, allowing remote users to connect and interact with the environment and with each other through their avatars, and with the real world using the virtual robots which are corresponding to real robots placed in the physical space!
Robots have clear implementation missions in such systems, facilitating presence of the remote users in the physical environment; provide opportunities for communication and cooperation with humans, enable actuation, teleoperation, and visual projection of the real world among other functions. This system enables users to see the real world through the eyes of the robots, use them as guides and to interact with the real environment, and even communicate with other visitors in the real physical space. These capabilities are moving the robots outside research labs to integrate them in the society with purpose and intent. The combination of robots, immersive technologies, computer graphics, wearables and other smart technologies as a fusion, can help to foster the ongoing digital transformation of our lives. The proposed CPSeS approach is used to bring life into the robots, which are not lifeless combination of hardware responding to programmable activities, but are embodying remote humans and carry them through the experience, being their remote actuation mechanism and the means of establishing communication with people in the real world.
The ‘humans within the robots’, can experience this combination of telepresence and tele-actuation through the immersive virtual environment, and take part in blended situations that can contribute into solving several issues pertaining accessibility, interaction, comprehensiveness and quality of experience.
At the recent STEM Centre for a Raspberry Jam 2020 event hosted by the British Computing Society (BCS – South Yorkshire Branch), the research team have conducted an initial pilot testing of the CPSeS to collect some initial reactions. The robot was navigating around at the event, and was broadcasting a live video feed during the presentations to a virtual reality mobile app, demonstrating the concept of connecting realities (Figure 3 and Figure 4)!
This CPSeS (Figure 5) is applicable to a wide range of socio-spatial domains with heterogeneous actors, for instance, businesses and their operations, industries, environmental changes, cultural heritage, education, psychophysical rehabilitation, search and rescue, field operations and other domains.
Using CPSeS to support gatherings, conferences, exhibitions, field trips, visits etc, can contribute towards containing virus spreads, reduce carbon footprint generated through travelling, include more participants and reach wider academic audience, mitigate the financial issues of travelling and accommodation, and other reasons hindering attendance. Transportation is one of the main factors impacting climate change. According to WHO: “Changes in choices and technologies can reduce transport’s major contributions to global CO2 emissions”, indicating that especially flying is the number one source of CO2 emissions. And considering the astonishing advancements in robotic and computer graphics technologies over the past few years, leveraging them can support our ongoing transformation to a digital society in several futuristic ways that until recently we thought impossible.
Introducing such Cyber-Physical-Social Eco-Society of systems provide opportunities that contribute into bringing people and spaces together in immersive and interactive ways! However there are several issues that need to be considered in order to support effective remote participation through the use of emerging technologies. Nevertheless, we believe that now is the time to begin embedding such technologies into a plethora of domains to help foster the digital transformation of the industry, the society and our lives in general.
