Hand Gesture Interface to Teach an Industrial Robots

Ahmadieh Khanesar, Mojtaba; Branson, David

Abstract

The present paper proposes user gesture recognition to control industrial robots. To recognize hand gestures, MediaPipe software package and an RGB camera is used. The proposed communication approach is an easy and reliable approach to provide commands for industrial robots. The landmarks which are extracted by MediaPipe software package are used as the input to a gesture recognition software to detect hand gestures. Five different hand gestures are recognized by the proposed machine learning approach in this paper. Hand gestures are then translated to movement directions for the industrial robot. The corresponding joint angle updates are generated using damped least squares inverse kinematic approach to move the industrial robot in a plane. The motion behaviour of the industrial robot is simulated within V-REP simulation environment. It is observed that the hand gestures are communicated with high accuracy to the industrial robot and the industrial robot follows the movements accurately.

• Presentation Conference Type: Conference Paper (unpublished)
• Conference Name: ICINCO 2023: 20th International Conference on Informatics in Control, Automation and Robotics
• Conference Location: Rome, Italy
• Start Date: Nov 13, 2023
• End Date: Nov 15, 2023
• Deposit Date: Nov 15, 2023
• Publicly Available Date: Nov 15, 2023
• Keywords: Image recognition, MediaPipe, gesture recognition, industrial robot control, inverse kinematics
• Public URL: https://nottingham-repository.worktribe.com/output/27373381
• Related Public URLs: https://icinco.scitevents.org/