Knowledge Commons of Institute of Automation,CAS
A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies | |
Xiaona Wang1,2![]() ![]() ![]() | |
发表期刊 | Robotic Intelligence and Automation
![]() |
ISSN | 2754-6969 |
2024 | |
卷号 | 44期号:2页码:316-333 |
通讯作者 | Chen, Jiahao(jiahao.chen@ia.ac.cn) |
文章类型 | 学术论文 |
摘要 | Purpose– Limited by the types of sensors, the state information available for musculoskeletal robots with highly redundant, nonlinear muscles is often incomplete, which makes the control face a bottleneck problem. The aim of this paper is to design a method to improve the motion performance of musculoskeletal robots in partially observable scenarios, and to leverage the ontology knowledge to enhance the algorithm’s adaptability to musculoskeletal robots that have undergone changes. Design/methodology/approach– A memory and attention-based reinforcement learning method is proposed for musculoskeletal robots with prior knowledge of muscle synergies. First, to deal with partially observed states available to musculoskeletal robots, a memory and attention-based network architecture is proposed for inferring more sufficient and intrinsic states. Second, inspired by muscle synergy hypothesis in neuroscience, prior knowledge of a musculoskeletal robot’s muscle synergies is embedded in network structure and reward shaping. Findings– Based on systematic validation, it is found that the proposed method demonstrates superiority over the traditional twin delayed deep deterministic policy gradients (TD3) algorithm. A musculoskeletal robot with highly redundant, nonlinear muscles is adopted to implement goal directed tasks. In the case of 21-dimensional states, the learning efficiency and accuracy are significantly improved compared with the traditional TD3 algorithm; in the case of 13-dimensional states without velocities and information from the end effector, the traditional TD3 is unable to complete the reaching tasks, while the proposed method breaks through this bottleneck problem. Originality/value– In this paper, a novel memory and attention-based reinforcement learning method with prior knowledge of muscle synergies is proposed for musculoskeletal robots to deal with partially observable scenarios. Compared with the existing methods, the proposed method effectively improves the performance. Furthermore, this paper promotes the fusion of neuroscience and robotics. |
关键词 | Musculoskeletal robot Partial observable Reinforcement learning LSTM Attention Muscle synergy |
DOI | 10.1108/RIA-11-2023-0172 |
收录类别 | SCIE |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[2021ZD0200408] ; Strategic Priority Research Program of Chinese Academy of Sciences[62203439] ; Strategic Priority Research Program of Chinese Academy of Sciences[91948303] ; Strategic Priority Research Program of Chinese Academy of Sciences[62173326] ; Major Program of the National Natural Science Foundation of China[XDB32050100] ; [T2293720] ; [T2293723] ; [T2293724] |
项目资助者 | National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Sciences ; Major Program of the National Natural Science Foundation of China |
WOS研究方向 | Automation & Control Systems ; Engineering |
WOS类目 | Automation & Control Systems ; Engineering, Manufacturing |
WOS记录号 | WOS:001199760500001 |
出版者 | EMERALD GROUP PUBLISHING LTD |
是否为代表性论文 | 是 |
七大方向——子方向分类 | 智能机器人 |
国重实验室规划方向分类 | 受人机理启发的类脑控制和类肌肉骨骼系统理论 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/57183 |
专题 | 多模态人工智能系统全国重点实验室_机器人理论与应用 |
通讯作者 | Jiahao Chen |
作者单位 | 1.中国科学院自动化研究所 2.中国科学院大学 |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Xiaona Wang,Jiahao Chen,Hong Qiao. A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies[J]. Robotic Intelligence and Automation,2024,44(2):316-333. |
APA | Xiaona Wang,Jiahao Chen,&Hong Qiao.(2024).A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies.Robotic Intelligence and Automation,44(2),316-333. |
MLA | Xiaona Wang,et al."A memory and attention-based reinforcement learning for musculoskeletal robots with prior knowledge of muscle synergies".Robotic Intelligence and Automation 44.2(2024):316-333. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
A memory and attenti(2591KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论