Knowledge Commons of Institute of Automation,CAS
Connecting Model-Based and Model-Free Control With Emotion Modulation in Learning Systems | |
Huang, Xiao1,2,3; Wu, Wei1,2,3; Qiao, Hong1,2,3,4,5 | |
发表期刊 | IEEE Transactions on Systems, Man, and Cybernetics: Systems |
ISSN | 2168-2216 |
2019-10-18 | |
卷号 | 10期号:4页码:1-15 |
通讯作者 | Qiao, Hong(hong.qiao@ia.ac.cn) |
摘要 | This article proposes a novel decision-making framework that bridges a gap between model-based (MB) and model-free (MF) control processes through only adjusting the planning horizon. Specifically, the output policy is obtained by solving a model predictive control problem with a locally optimal state value as terminal constraints. When the planning horizon decreases to zero, the MB control will transform into the MF control smoothly. Meanwhile, inspired by the neural mechanism of emotion modulation on decision-making, we build a biologically plausible computational model of emotion processing. This model can generate an uncertainty-related emotional response on the basis of the state prediction error and reward prediction error, and then dynamically modulates the planning horizon in the tasks. The simulation results demonstrate that the proposed decision-making framework can produce better policies than traditional methods. Emotion modulation can shift the MB and MF control well to improve the learning efficiency and the speed of decision-making. |
关键词 | Brain-inspired computing decision-making emotion modulation emotion-cognition interactions reinforcement learning |
DOI | 10.1109/TSMC.2019.2933152 |
关键词[WOS] | BASAL GANGLIA ; AMYGDALA ; UNCERTAINTY ; STRIATUM ; NEURONS ; ROBOTS ; ROLES |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Strategic Priority Research Program of Chinese Academy of Science[XDB32050100] ; National Key Research and Development Program of China[2017YFB1300203] ; National Key Research and Development Program of China[2017YFB1300200] ; National Natural Science Foundation of China[61702516] ; Development of Science and Technology of Guangdong Province Special Fund Project[2016B090910001] ; National Natural Science Foundation of China[91648205] ; National Natural Science Foundation of China[61627808] ; National Natural Science Foundation of China[61627808] ; National Natural Science Foundation of China[91648205] ; Development of Science and Technology of Guangdong Province Special Fund Project[2016B090910001] ; National Natural Science Foundation of China[61702516] ; National Key Research and Development Program of China[2017YFB1300200] ; National Key Research and Development Program of China[2017YFB1300203] ; Strategic Priority Research Program of Chinese Academy of Science[XDB32050100] |
项目资助者 | National Key Research and Development Program of China ; National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Science ; Development of Science and Technology of Guangdong Province Special Fund Project |
WOS研究方向 | Automation & Control Systems ; Computer Science |
WOS类目 | Automation & Control Systems ; Computer Science, Cybernetics |
WOS记录号 | WOS:000673624500002 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 类脑模型与计算 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/39095 |
专题 | 多模态人工智能系统全国重点实验室_机器人理论与应用 |
通讯作者 | Qiao, Hong |
作者单位 | 1.State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China 2.Beijing Key Laboratory of Research and Application for Robotic Intelligence of Hand-Eye-Brain Interaction, Beijing 100190, China 3.University of Chinese Academy of Sciences, Beijing 100049, China 4.CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing 100190, China 5.Cloud Computing Center, Chinese Academy of Sciences, Beijing 100190, China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Huang, Xiao,Wu, Wei,Qiao, Hong. Connecting Model-Based and Model-Free Control With Emotion Modulation in Learning Systems[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems,2019,10(4):1-15. |
APA | Huang, Xiao,Wu, Wei,&Qiao, Hong.(2019).Connecting Model-Based and Model-Free Control With Emotion Modulation in Learning Systems.IEEE Transactions on Systems, Man, and Cybernetics: Systems,10(4),1-15. |
MLA | Huang, Xiao,et al."Connecting Model-Based and Model-Free Control With Emotion Modulation in Learning Systems".IEEE Transactions on Systems, Man, and Cybernetics: Systems 10.4(2019):1-15. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Connecting Model-bas(1614KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论