Knowledge Commons of Institute of Automation,CAS
Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning | |
Zhang TL(张天乐)1,2![]() ![]() ![]() ![]() | |
发表期刊 | International Journal of Control, Automation and Systems
![]() |
2023 | |
页码 | 1-13 |
文章类型 | 研究性论文 |
摘要 | Although deep reinforcement learning has recently achieved some successes in robot navigation, there are still unsolved problems. Particularly, a robot guided by a distant ultimate goal is easy to get stuck in danger or encounter collisions in dynamic crowded environments due to the lack of long-term perspectives. In this paper, a novel subgoal-guided approach based on two-level hierarchical deep reinforcement learning with spatial-temporal graph attention networks (ST-GANets), called SG-HDRL, is proposed for a robot navigating in a dynamic crowded environment with autonomous obstacles, e.g., crowd. Specifically, the high-level policy, that models the spatial-temporal relation between the robot and the obstacles using the obstacles’ trajectories by the designed high-level ST-GANet, generates intermediate subgoals from a longer-term perspective over higher temporal scales. The sub-goals give a favorable and collision-free direction to avoid encountering danger or collisions while approaching the ultimate goal. The low-level policy, that similarly implements the designed low-level ST-GANet to implicitly predict the obstacles’ motions, takes the subgoals as short-term guidance through an intrinsic reward incentive to generate primitive actions for the robot. Simulation results demonstrate that SG-HDRL using ST-GANets has better performances compared with state-of-the-art baselines. Furthermore, the proposed SG-HDRL is deployed to an experimental platform based on omnidirectional cars, and experiment results validate the effectiveness and practicability of the proposed SG-HDRL. |
收录类别 | SCI |
语种 | 英语 |
WOS记录号 | WOS:000982815800004 |
七大方向——子方向分类 | 智能机器人 |
国重实验室规划方向分类 | 多智能体决策 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/51958 |
专题 | 复杂系统认知与决策实验室_飞行器智能技术 |
通讯作者 | Liu Z(刘振) |
作者单位 | 1.中国科学院自动化研究所 2.中国科学院大学人工智能学院 3.澳门科技大学 |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhang TL,Liu Z,Pu ZQ,et al. Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning[J]. International Journal of Control, Automation and Systems,2023:1-13. |
APA | Zhang TL,Liu Z,Pu ZQ,Yi JQ,Liang YY,&Zhang D.(2023).Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning.International Journal of Control, Automation and Systems,1-13. |
MLA | Zhang TL,et al."Robot Subgoal-guided Navigation in Dynamic Crowded Environments with Hierarchical Deep Reinforcement Learning".International Journal of Control, Automation and Systems (2023):1-13. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
2022-IJCAS.pdf(6472KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论