NSF CAREER

It is to develop adaptive human multi-robot systems.

This project develops adaptive human multi-robot systems that can flexibly respond to changes in situation and task needs. It develops methods for real-time monitoring and analysis of the cognitive and emotional state of operators, enabling human operators to adapt to robot system changes and robots to adapt to human cognitive and emotional states. By developing adaptive systems to improve the performance of human-robot teams, the project advances our understanding of human multi-robot interactions. The new technologies provided will improve function of human multi-robot teams deployed (for example) in environmental monitoring, nuclear cleanup, disaster response, and defense. The project develops novel concepts and tools for

Designing mission-specific human multi-robot teams; Estimating operator cognitive and emotional state from physiological and behavioral signals; and Enabling adaptive control of autonomy, tasks, and resource assignment. This approach will enhance the fields of multi-robot systems and human-robot interactions and improve performance and versatility of human-robot teams. The project will increase our understanding of how multiple robots can more effectively interact with multiple humans and vice versa, and advance the scale and capability of human multi-robot systems.

The project also advances STEM education and workforce development by involving K-12 students, undergraduate and graduate women, minorities, and underrepresented groups in human-robot interaction and multi-robot systems.

Dr. Byung-Cheol Min, Associate Professor in the Department of Computer and Information Technology and Director of the Purdue SMART Lab, is the principal investigator of this project. This material is based upon work supported by the National Science Foundation under Grant No. IIS-1846221. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

1. ROSbag-based Affective Dataset

We release a new rosbag-based multimodal affective dataset for emotional and cognitive states generated using the Robot Operating System (ROS). In order to collect physiological and behavioral dataset from human participants, we developed this ROS system including commercial wearable sensors: Empatica E4, Shimmer3 GSR and PPG, Polar H10, and Myo armband. Also, we added following ehavioral sensors to the ROS system: a USB camera (for side view), Intel RealSense (frontal depth and RGB images), a microphone, and a mouse & keyboard. As a results, the generated affective dataset consists of 1,602 ROSbag files, and the size of the dataset is about 787GB. The dataset is made publicly available. We expect that our dataset can be a great resource for many researchers in the fields of affective computing, Human-Computer Interaction (HCI), and Human-Robot Interaction (HRI).​​

2. An Adaptive Multi-human Multi-robot System Framework based on Individual Human and Robot Condition and Performance

Multi-human multi-robot (MH-MR) systems have the ability to combine the potential advantages of robotic systems and humans in the loop. Robotic systems contribute with precision performance and long operation of repetitive tasks without tire, while humans in the loop enhance decision making abilities following improved situational awareness. The system’s ability to adapt to changing conditions and performance of each individual (humans and robots) during the mission is vital to maintaining overall system performance. Given the variety and operational scale of such a system, development of a generalized framework is pertinent. The research goal is to develop a generalized MH-MR system framework capable of allocating the system workload adaptively to health conditions and work performance of human operated and autonomous robots. The framework consists of removable modular blocks ensuring applicability to different MH-MR scenarios. A new workload transition block ensures smooth transition without adverse affects of the workload change on the individual agents

  • Video: https://youtu.be/-WY49FPbNWg
  • Papers: Tamzidul Mina, Shyam Sundar Kannan, Wonse Jo, and Byung-Cheol Min, "Adaptive Workload Allocation for Multi-human Multi-robot Teams for Independent and Homogeneous Tasks", IEEE Access, Vol. 8, pp. 152697-152712, 2020; Link​​​