A User Study Evaluation of Predictive Formal Modelling at Runtime in Human-Swarm Interaction

Abioye, Ayodeji O.; Hunt, William; Gu, Yue; Schneiders, Eike; Naiseh, Mohammad; Archibald, Blair; Sevegnani, Michele; Ramchurn, Sarvapali D.; Fischer, Joel E.; Soorati, Mohammad D.

Abstract

Formal Modelling is often used as part of the design and testing process of software development to ensure that components operate within suitable bounds even in unexpected circumstances. We conducted a user study evaluation of predictive formal modelling (PFM) at runtime in a human-swarm mission to determine the benefit of predictive formal modelling on performance and human-swarm interaction. 180 participants were recruited to perform the role of aerial swarm operators delivering parcels to target locations in a simulation environment. The PFM model was integrated into the simulation software to inform the operator of the estimated mission completion time given the current number of drones deployed. The operator could increase the number of parcels delivered in any time step by adding drones, which also increased costs, thus requiring the use of the minimum number of drones necessary to complete the task in the given time. We collected user feedback using standard survey questionnaires and measured performance using data obtained from the Human And Robot Interactive Swarm (HARIS) simulator. Our results show that PFM increased the performance of the human swarm team without significantly increasing the operators’ workload or affecting the system's usability.

Citation

Abioye, A. O., Hunt, W., Gu, Y., Schneiders, E., Naiseh, M., Archibald, B., Sevegnani, M., Ramchurn, S. D., Fischer, J. E., & Soorati, M. D. (in press). A User Study Evaluation of Predictive Formal Modelling at Runtime in Human-Swarm Interaction. ACM Transactions on Human-Robot Interaction, https://doi.org/10.1145/3727989

• Journal Article Type: Article
• Acceptance Date: Mar 28, 2025
• Online Publication Date: Apr 3, 2025
• Deposit Date: Apr 7, 2025
• Publicly Available Date: Apr 7, 2025
• Journal: ACM Transactions on Human-Robot Interaction
• Print ISSN: 2573-9522
• Electronic ISSN: 2573-9522
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1145/3727989
• Public URL: https://nottingham-repository.worktribe.com/output/47543937
• Publisher URL: https://dl.acm.org/doi/10.1145/3727989
• Additional Information: Received: 2024-05-06; Accepted: 2025-03-28; Published: 2025-04-03