CASIA OpenIR  > 类脑智能研究中心
A Brain-Inspired Decision Making Model Based on Top-Down Biasing of Prefrontal Cortex to Basal Ganglia and Its Application in Autonomous UAV Explorations
Zhao Feifei1,3; Zeng Yi1,2,3; Wang Guixiang1; Bai Jun1; Xu Bo1,2,3
Source PublicationCognitive Computation
2018
Volume10Issue:2Pages:296-306
Subtype期刊论文
Abstract

Decision making is a fundamental ability for intelligent agents (e.g., humanoid robots and unmanned aerial vehicles). During decision making process, agents can improve the strategy for interacting with the dynamic
environment through reinforcement learning. Many state-of-the-art reinforcement learning models deal with relatively smaller number of state-action pairs, and the states are preferably discrete, such as Q-learning and Actor-Critic algorithms. While in practice, in many scenario, the states are continuous and hard to be properly discretized. Better autonomous decision making methods need to be proposed to handle these problems. Inspired by the mechanism of decision making in human brain, we propose a general computational model, named as prefrontal cortex-basal ganglia (PFC-BG) algorithm. The proposed model is inspired by the biological reinforcement learning pathway and mechanisms from the following perspectives: (1) Dopamine signals continuously update reward-relevant information for both basal ganglia and working memory in prefrontal cortex. (2) We maintain the contextual reward information in working memory. This has a top-down biasing effect on reinforcement learning in basal ganglia. The proposed model separates the continuous states into smaller distinguishable states, and introduces continuous reward function for each state to obtain reward information at different time. To verify the performance of our model, we apply it to many UAV decision making experiments, such as avoiding obstacles and flying through window and door, and the experiments support the effectiveness of the model. Compared with traditional Q-learning and Actor-Critic algorithms, the proposed model is more biologically inspired, and more accurate and faster to make decision.

KeywordPrefrontal Cortex Working Memory Basal Ganglia Dopamine System Brain-inspired Decision Making Model
Indexed BySCI
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/23556
Collection类脑智能研究中心
Corresponding AuthorZeng Yi
Affiliation1.Institute of Automation, Chinese Academy of Sciences, Beijing, China
2.Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
3.University of Chinese Academy of Sciences, Beijing, China
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Corresponding Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Zhao Feifei,Zeng Yi,Wang Guixiang,et al. A Brain-Inspired Decision Making Model Based on Top-Down Biasing of Prefrontal Cortex to Basal Ganglia and Its Application in Autonomous UAV Explorations[J]. Cognitive Computation,2018,10(2):296-306.
APA Zhao Feifei,Zeng Yi,Wang Guixiang,Bai Jun,&Xu Bo.(2018).A Brain-Inspired Decision Making Model Based on Top-Down Biasing of Prefrontal Cortex to Basal Ganglia and Its Application in Autonomous UAV Explorations.Cognitive Computation,10(2),296-306.
MLA Zhao Feifei,et al."A Brain-Inspired Decision Making Model Based on Top-Down Biasing of Prefrontal Cortex to Basal Ganglia and Its Application in Autonomous UAV Explorations".Cognitive Computation 10.2(2018):296-306.
Files in This Item: Download All
File Name/Size DocType Version Access License
Zhao2018_Article_ABr(3853KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Zhao Feifei]'s Articles
[Zeng Yi]'s Articles
[Wang Guixiang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Zhao Feifei]'s Articles
[Zeng Yi]'s Articles
[Wang Guixiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Zhao Feifei]'s Articles
[Zeng Yi]'s Articles
[Wang Guixiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Zhao2018_Article_ABrain-InspiredDecisionMakingM.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.