Intent-Aware Predictive Haptic Guidance and its Application to Shared Control Teleoperation

Ly, Kim Tien; Poozhiyil, Mithun; Pandya, Harit; Neumann, Gerhard; Kucukyilmaz, Ayse

Abstract

This paper presents a haptic shared control paradigm that modulates the level of robotic guidance, based on predictions of human motion intentions. The proposed method incorporates robot trajectories learned from human demonstrations and dynamically adjusts the level of robotic assistance based on how closely the detected intentions match these trajectories. An experimental study is conducted to demonstrate the paradigm on a teleoperated pick-and-place task using a Franka Emika Panda robot arm, controlled via a 3D Systems Touch X haptic interface. In the experiment, the human operator teleoperates a remote robot arm while observing the environment on a 2D screen. While the human teleoperates the robot arm, the objects are tracked, and the human's motion intentions (e.g., which object will be picked or which bin will be approached) are predicted using a Deep Q-Network (DQN). The predictions are made considering the current robot state and baseline robot trajectories that are learned from human demonstrations using Probabilistic Movement Primitives (ProMPs). The detected intentions are then used to condition the ProMP trajectories to modulate the movement and accommodate changing object configurations. Consequently, the system generates adaptive force guidance as weighted virtual fixtures that are rendered on the haptic device. The outcomes of the user study, conducted with 12 participants, indicate that the proposed paradigm can successfully guide users to robust grasping configurations and brings better performance by reducing the number of grasp attempts and improving trajectory smoothness and length.

• Conference Name: 2021 30th IEEE International Conference on Robot and Human Interactive Communication, RO-MAN 2021
• Conference Location: Vancouver, Canada (Online)
• Start Date: Aug 8, 2021
• End Date: Aug 12, 2021
• Acceptance Date: Jun 11, 2021
• Online Publication Date: Aug 23, 2021
• Publication Date: Aug 8, 2021
• Deposit Date: Jul 7, 2021
• Publicly Available Date: Aug 8, 2021
• Publisher: IEEE
• Pages: 565-572
• Series Title: International Symposium on Robot and Human Interactive Communication
• Series Number: 30
• Series ISSN: 1944-9445
• Book Title: 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)
• ISBN: 9781665404921
• DOI: https://doi.org/10.1109/RO-MAN50785.2021.9515326
• Keywords: Performance evaluation , Adaptive systems , Three-dimensional displays , Tracking , Navigation , Grasping , Manipulators
• Public URL: https://nottingham-repository.worktribe.com/output/5721206
• Publisher URL: https://ieeexplore.ieee.org/document/9515326
• Related Public URLs: https://ro-man2021.org/