A Visual-Based Gesture Prediction Framework Applied in Social Robots

doi:10.1109/JAS.2021.1004243

CASIA OpenIR > 学术期刊 > IEEE/CAA Journal of Automatica Sinica

	A Visual-Based Gesture Prediction Framework Applied in Social Robots
	Bixiao Wu; Junpei Zhong; Chenguang Yang
发表期刊	IEEE/CAA Journal of Automatica Sinica
ISSN	2329-9266
	2022
卷号	9 期号:3 页码:510-519
摘要	In daily life, people use their hands in various ways for most daily activities. There are many applications based on the position, direction, and joints of the hand, including gesture recognition, gesture prediction, robotics and so on. This paper proposes a gesture prediction system that uses hand joint coordinate features collected by the Leap Motion to predict dynamic hand gestures. The model is applied to the NAO robot to verify the effectiveness of the proposed method. First of all, in order to reduce jitter or jump generated in the process of data acquisition by the Leap Motion, the Kalman filter is applied to the original data. Then some new feature descriptors are introduced. The length feature, angle feature and angular velocity feature are extracted from the filtered data. These features are fed into the long-short time memory recurrent neural network (LSTM-RNN) with different combinations. Experimental results show that the combination of coordinate, length and angle features achieves the highest accuracy of 99.31%, and it can also run in real time. Finally, the trained model is applied to the NAO robot to play the finger-guessing game. Based on the predicted gesture, the NAO robot can respond in advance.
关键词	Finger-guessing game gesture prediction human-robot interaction long-short time memory recurrent neural network (LSTM-RNN) social robot
DOI	10.1109/JAS.2021.1004243
引用统计	被引频次：17[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/47211
专题	学术期刊_IEEE/CAA Journal of Automatica Sinica
推荐引用方式 GB/T 7714	Bixiao Wu,Junpei Zhong,Chenguang Yang. A Visual-Based Gesture Prediction Framework Applied in Social Robots[J]. IEEE/CAA Journal of Automatica Sinica,2022,9(3):510-519.
APA	Bixiao Wu,Junpei Zhong,&Chenguang Yang.(2022).A Visual-Based Gesture Prediction Framework Applied in Social Robots.IEEE/CAA Journal of Automatica Sinica,9(3),510-519.
MLA	Bixiao Wu,et al."A Visual-Based Gesture Prediction Framework Applied in Social Robots".IEEE/CAA Journal of Automatica Sinica 9.3(2022):510-519.

条目包含的文件
文件名称/大小	文献类型	版本类型	开放类型	使用许可
JAS-2021-0367.pdf（37924KB）	期刊论文	出版稿	开放获取	CC BY-NC-SA	浏览下载