Knowledge Commons of Institute of Automation,CAS
A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat | |
Jiajun Chai; Wenzhang Chen; Yuanheng Zhu; Zong-xin Yao,; Dongbin Zhao | |
发表期刊 | IEEE Transactions on Systems, Man and Cybernetics: Systems |
2023 | |
页码 | DOI: 10.1109/TSMC.2023.3270444 |
摘要 | Unmanned combat air vehicle (UCAV) combat is a challenging scenario with high-dimensional continuous state and action space and highly nonlinear dynamics. In this paper, we propose a general hierarchical framework to resolve the within-vision-range (WVR) air-to-air combat problem under six dimensions of degree (6-DOF) dynamics. The core idea is to divide the whole decision-making process into two loops and use reinforcement learning (RL) to solve them separately. The outer loop uses a combat policy to decide the macro command according to the current combat situation. Then the inner loop uses a control policy to answer the macro command by calculating the actual input signals for the aircraft. We design the Markov decision-making process for the control policy and the Markov game between two aircraft. We present a twostage training mechanism. For the control policy, we design an effective reward function to accurately track various macro behaviors. For the combat policy, we present a fictitious self-play mechanism to improve the combat performance by combating against the historical combat policies. Experiment results show that the control policy can achieve better tracking performance than conventional methods. The fictitious self-play mechanism can learn competitive combat policy, which can achieve high winning rates against conventional methods. |
WOS记录号 | WOS:000988479700001 |
七大方向——子方向分类 | 机器博弈 |
国重实验室规划方向分类 | 开放博弈基础理论 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/51548 |
专题 | 多模态人工智能系统全国重点实验室_深度强化学习 |
通讯作者 | Wenzhang Chen |
推荐引用方式 GB/T 7714 | Jiajun Chai,Wenzhang Chen,Yuanheng Zhu,et al. A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat[J]. IEEE Transactions on Systems, Man and Cybernetics: Systems,2023:DOI: 10.1109/TSMC.2023.3270444. |
APA | Jiajun Chai,Wenzhang Chen,Yuanheng Zhu,Zong-xin Yao,,&Dongbin Zhao.(2023).A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat.IEEE Transactions on Systems, Man and Cybernetics: Systems,DOI: 10.1109/TSMC.2023.3270444. |
MLA | Jiajun Chai,et al."A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat".IEEE Transactions on Systems, Man and Cybernetics: Systems (2023):DOI: 10.1109/TSMC.2023.3270444. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
main.pdf(9249KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论