Bringing Adaptive and Immersive Interfaces to Real-World Multi-Robot Scenarios: Application to Surveillance and Intervention in Infrastructures

Our paper "Bringing Adaptive and Immersive Interfaces to Real-World Multi-Robot Scenarios: Application to Surveillance and Intervention in Infrastructures" has been published by IEEE Access!

Multiple robot missions imply a series of challenges for single human operators, such as managing high workloads or maintaining a correct level of situational awareness. Conventional interfaces are not prepared to face these challenges; however, new concepts have arisen to cover this need, such as adaptive and immersive interfaces. This paper reports the design and development of an adaptive and immersive interface, as well as a complete set of experiments carried out to establish comparisons with a conventional one. The interface object of study has been developed using virtual reality to bring operators into scenarios and allow an intuitive commanding of robots. Additionally, it is able to recognize the mission’s state and show hints to the operators. The experiments were performed in both outdoor and indoor scenarios recreating an intervention after an accident in critical infrastructure. The results show the potential of adaptive and immersive interfaces in the improvement of workload, situational awareness and performance of operators in multi-robot missions.

I'm very proud to see the result of many days of hard work during the past year, and I want to thank my friends Elena and Pablo for their help to make this real.

J.J. Roldán, E. Peña-Tapia, P. Garcia-Aunon, J. del Cerro and A. Barrientos. Bringing adaptive & immersive interfaces to real-world multi-robot scenarios: application to surveillance and intervention in infrastructures. IEEE Access, 7 (1), 86319-86335, 2019. Impact Factor (JCR, 2018): 4.098, Q1. Article

Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame

The paper "Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame" has been published by Robotics, an open access journal published by MDPI and obviously focused on robotics.

One of the active challenges in multi-robot missions is related to managing operator workload and situational awareness. Currently, the operators are trained to use interfaces, but in the near future this can be turned inside out: the interfaces will adapt to operators so as to facilitate their tasks. To this end, the interfaces should manage models of operators and adapt the information to their states and preferences. This work proposes a videogame-based approach to classify operator behavior and predict their actions in order to improve teleoperated multi-robot missions. First, groups of operators are generated according to their strategies by means of clustering algorithms. Second, the operators’ strategies are predicted, taking into account their models. Multiple information sources and modeling methods are used to determine the approach that maximizes the mission goal. The results demonstrate that predictions based on previous data from single operators increase the probability of success in teleoperated multi-robot missions by 19%, whereas predictions based on operator clusters increase this probability of success by 28%. 

J11: J.J. Roldán, V. Díaz-Maroto, J. Real, P.R. Palafox, J. Valente, M. Garzón and A. Barrientos. Press start to play: classifying multi-robot operators and predicting their strategies through a videogame. Robotics, 8 (3), 53-67, 2019. Impact Factor (Scopus, 2018): 1.53, Q2. Article