Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

doi:10.1109/TNNLS.2021.3054402

	Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning
	Wang, Yu1 ; Tang, Chong2,3 ; Wang, Shuo1,4,5 ; Cheng, Long1 ; Wang, Rui1 ; Tan, Min1,4 ; Hou, Zengguang1
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
	2021-02-08
页码	12
通讯作者	Cheng, Long(long.cheng@ia.ac.cn)
摘要	In this article, the underwater target tracking control problem of a biomimetic underwater vehicle (BUV) is addressed. Since it is difficult to build an effective mathematic model of a BUV due to the uncertainty of hydrodynamics, target tracking control is converted into the Markov decision process and is further achieved via deep reinforcement learning. The system state and reward function of underwater target tracking control are described. Based on the actor-critic reinforcement learning framework, the deep deterministic policy gradient actor-critic algorithm with supervision controller is proposed. The training tricks, including prioritized experience replay, actor network indirect supervision training, target network updating with different periods, and expansion of exploration space by applying random noise, are presented. Indirect supervision training is designed to address the issues of low stability and slow convergence of reinforcement learning in the continuous state and action space. Comparative simulations are performed to show the effectiveness of the training tricks. Finally, the proposed actor-critic reinforcement learning algorithm with supervision controller is applied to the physical BUV. Swimming pool experiments of underwater object tracking of the BUV are conducted in multiple scenarios to verify the effectiveness and robustness of the proposed method.
关键词	Reinforcement learning Target tracking Robots Sports Aerospace electronics Mobile robots Underwater vehicles Biomimetic underwater vehicle (BUV) reinforcement learning target tracking control
DOI	10.1109/TNNLS.2021.3054402
关键词[WOS]	MOVING-TARGET ; MOBILE ROBOT
收录类别	SCI
语种	英语
资助项目	Youth Innovation Promotion Association CAS[2018162] ; National Natural Science Foundation of China[U1713222] ; National Natural Science Foundation of China[62025307] ; National Natural Science Foundation of China[62073316] ; National Natural Science Foundation of China[U1806204] ; National Natural Science Foundation of China[62033013] ; Beijing Municipal Natural Science Foundation[JQ19020]
项目资助者	Youth Innovation Promotion Association CAS ; National Natural Science Foundation of China ; Beijing Municipal Natural Science Foundation
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000732356900001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
七大方向——子方向分类	智能控制
引用统计	被引频次：23[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/46849
专题	复杂系统认知与决策实验室_先进机器人
通讯作者	Cheng, Long
作者单位	1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.NUCTECH Co Ltd, Beijing 100084, Peoples R China 3.Tsinghua Univ, Dept Comp Sci & Technol, Beijing 100084, Peoples R China 4.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 5.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai 200031, Peoples R China
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Wang, Yu,Tang, Chong,Wang, Shuo,et al. Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2021:12.
APA	Wang, Yu.,Tang, Chong.,Wang, Shuo.,Cheng, Long.,Wang, Rui.,...&Hou, Zengguang.(2021).Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,12.
MLA	Wang, Yu,et al."Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2021):12.