A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle

doi:10.1109/TCDS.2017.2649564

	A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle
	Zeng, Yi1,2 ; Wang, Guixiang1 ; Xu, Bo1,2
发表期刊	IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS
ISSN	2379-8920
	2018-06-01
卷号	10 期号:2 页码:290-303
通讯作者	Zeng, Yi(yi.zeng@ia.ac.cn)
摘要	Reinforcement learning brings flexibility and generality for machine learning, while most of them are mathematical optimization driven approaches, and lack of cognitive and neural evidence. In order to provide a more cognitive and neural mechanisms driven foundation and validate its applicability in complex task, we develop a basal ganglia (BG) network centric reinforcement learning model. Compared to existing work on modeling BG, this paper is unique from the following perspectives: 1) the orbitofrontal cortex (OFC) is taken into consideration. OFC is critical in decision making because of its responsibility for reward representation and is critical in controlling the learning process, while most of the BG centric models do not include OFC; 2) to compensate the inaccurate memory of numeric values, precise encoding is proposed to enable working memory system remember important values during the learning process. The method combines vector convolution and the idea of storage by digit bit and is efficient for accurate value storage; and 3) for information coding, the Hodgkin-Huxley model is used to obtain a more biological plausible description of action potential with plenty of ionic activities. To validate the effectiveness of the proposed model, we apply the model to the unmanned aerial vehicle (UAV) autonomous learning process in a 3-D environment. Experimental results show that our model is able to give the UAV the ability of free exploration in the environment and has comparable learning speed as the Q learning algorithm, while the major advances for our model is that it is with solid cognitive and neural basis.
关键词	Basal ganglia (BG) network brain-inspired intelligence precise encoding reinforcement learning model unmanned aerial vehicle (UAV) autonomous learning
DOI	10.1109/TCDS.2017.2649564
关键词[WOS]	ORBITOFRONTAL CORTEX ; FUNCTIONAL-ANATOMY ; DECISION-MAKING ; BRAIN ; CIRCUITS
收录类别	SCI
语种	英语
资助项目	Chinese Academy of Sciences[XDB02060007] ; Beijing Municipal Commission of Science and Technology[Z151100000915070] ; Beijing Municipal Commission of Science and Technology[Z161100000216124] ; Chinese Academy of Sciences[XDB02060007] ; Beijing Municipal Commission of Science and Technology[Z151100000915070] ; Beijing Municipal Commission of Science and Technology[Z161100000216124]
项目资助者	Chinese Academy of Sciences ; Beijing Municipal Commission of Science and Technology
WOS研究方向	Computer Science ; Robotics ; Neurosciences & Neurology
WOS类目	Computer Science, Artificial Intelligence ; Robotics ; Neurosciences
WOS记录号	WOS:000435198600015
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：9[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/27938
专题	脑图谱与类脑智能实验室_类脑认知计算
通讯作者	Zeng, Yi
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Shanghai 200031, Peoples R China
第一作者单位	中国科学院自动化研究所
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Zeng, Yi,Wang, Guixiang,Xu, Bo. A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle[J]. IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS,2018,10(2):290-303.
APA	Zeng, Yi,Wang, Guixiang,&Xu, Bo.(2018).A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle.IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS,10(2),290-303.
MLA	Zeng, Yi,et al."A Basal Ganglia Network Centric Reinforcement Learning Model and Its Application in Unmanned Aerial Vehicle".IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS 10.2(2018):290-303.