Multimodal Human-Robot Interface for Accessible Remote Robotic Interventions in Hazardous Environments

Abstract

Human-Robot Interfaces have a key role in the design of secure and efficient robotic systems. Great effort has been put during the past decades on the design of advanced interfaces for domestic and industrial robots. However, robots for intervention in unplanned and hazardous scenarios still need further research, especially when the mission requires the use of multiple robotic systems, to obtain an acceptable level of usability and safety. This paper describes the design and the software engineering process behind the development of a modular and multimodal Human-Robot Interface for intervention with a cooperative team of robots, as well as its validation and commissioning, as it is being used in real operations at CERN's accelerators complex. The proposed Human-Robot Interface allows the control of a heterogeneous set of robots homogeneously, providing the operator, among other features, with live scripting functionalities which can be programmed and adapted in run-time, for example, to increase operator's multi-tasking in a multi-agent scenario. The operator is given the capability to enter in the control loop between the HRI and the robot and customize the control commands according to the operation. To provide such functionalities, well-defined software development approaches have been adopted, for guaranteeing the modularity and the safety of the system during its continuous development. The paper describes the modules offered by the HRI, such as the multimodality, multi-robot control, safety, operators training, and communications architecture, among others. The HRI and the CERN Robotic Framework where it belongs are designed in a modular manner, in order to be able to adapt both, software and hardware architecture in a short time, to the next planned mission. Results present the experience gained with the system, demonstrating a high level of usability, learnability and safety when operated by both, non-experts and qualified robotic operators. The multimodal user interface has demonstrated to be very accurate and secure, providing a unique system to control, in a teleoperated or supervised manner, both single and multiple heterogeneous mobile manipulators. At the moment of writing, the user interface has been successfully used in 100 real interventions in radioactive industrial environments. The presented HRI is a novel research contribution in terms of multimodality, adaptability and modularity for mobile manipulator robotic teams in radioactive environments, especially for its software architecture, as part of the CERN Robotic Framework.