Video-based prediction of hand-grasp preshaping with application to prosthesis control

Taverne, Luke T. (ETH Zurich, Switzerland) ; Cognolato, Matteo (University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis) ; ETH Zurich, Switzerland) ; Bützer, Tobias (ETH Zurich, Switzerland) ; Gassert, Roger (ETH Zurich, Switzerland) ; Hilliges, Otmar (ETH Zurich, Switzerland)

Among the currently available grasp-type selection techniques for hand prostheses, there is a distinct lack of intuitive, robust, low-latency solutions. In this paper we investigate the use of a portable, forearm-mounted, video-based technique for the prediction of hand-grasp preshaping for arbitrary objects. The purpose of this system is to automatically select the grasp-type for the user of the prosthesis, potentially increasing ease-of-use and functionality. This system can be used to supplement and improve existing control strategies, such as surface electromyography (sEMG) pattern recognition, for prosthetic and orthotic devices. We designed and created a suitable dataset consisting of RGB-D video data for 2212 grasp examples split evenly across 7 classes; 6 grasps commonly used in activities of daily living, and an additional no-grasp category. We processed and analyzed the dataset using several state-of-the-art deep learning architectures. Our selected model shows promising results for realistic, intuitive, real-world use, reaching per-frame accuracies on video sequences of up to 95.90% on the validation set. Such a system could be integrated into the palm of a hand prosthesis, allowing an automatic prediction of the grasp-type without requiring any special movements or aiming by the user.


Keywords:
Conference Type:
full paper
Faculty:
Economie et Services
School:
HEG-VS
Institute:
Institut Informatique de gestion
Subject(s):
Informatique
Publisher:
Montreal, Canada, 20-24 May 2019
Date:
2019-05
Montreal, Canada
20-24 May 2019
Pagination:
Pp. 4975-4982
Published in:
Proceedings of 2019 International Conference on Robotics and Automation (ICRA)
ISBN:
978-1-5386-6027-0
Appears in Collection:

Note: The status of this file is: restricted


 Record created 2020-06-18, last modified 2020-06-19

Fulltext:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)