| Discrete soft actor-critic with auto-encoder on vascular robotic system | |
Li, Hao1,2; Zhou, Xiao-Hu1,2 ; Xie, Xiao-Liang1,2; Liu, Shi-Qi1,2; Gui, Mei-Jiang1,2; Xiang, Tian-Yu1,2; Wang, Jin-Li3; Hou, Zeng-Guang1,2,4
| |
| Source Publication | ROBOTICA
 |
| ISSN | 0263-5747 |
| 2022-11-17 | |
| Pages | 12 |
| Corresponding Author | Zhou, Xiao-Hu(xiaohu.zhou@ia.ac.cn) ; Xie, Xiao-Liang(xiaoliang.xie@ia.ac.cn) |
| Abstract | Instrument delivery is critical part in vascular intervention surgery. Due to the soft-body structure of instruments, the relationship between manipulation commands and instrument motion is non-linear, making instrument delivery challenging and time-consuming. Reinforcement learning has the potential to learn manipulation skills and automate instrument delivery with enhanced success rates and reduced workload of physicians. However, due to the sample inefficiency when using high-dimensional images, existing reinforcement learning algorithms are limited on realistic vascular robotic systems. To alleviate this problem, this paper proposes discrete soft actor-critic with auto-encoder (DSAC-AE) that augments SAC-discrete with an auxiliary reconstruction task. The algorithm is applied with distributed sample collection and parameter update in a robot-assisted preclinical environment. Experimental results indicate that guidewire delivery can be automatically implemented after 50k sampling steps in less than 15 h, demonstrating the proposed algorithm has the great potential to learn manipulation skill for vascular robotic systems. |
| Keyword | surgical robots vascular robotic system automation reinforcement learning deep neural network |
| DOI | 10.1017/S0263574722001527 |
| WOS Keyword | INTERVENTIONAL CARDIOLOGISTS ; STAFF ; RISK |
| Indexed By | SCI |
| Language | 英语 |
| Funding Project | National Natural Science Foundation of China[62003343] ; National Natural Science Foundation of China[62222316] ; National Natural Science Foundation of China[U1913601] ; National Natural Science Foundation of China[62073325] ; National Natural Science Foundation of China[61720106012] ; National Natural Science Foundation of China[U20A20224] ; National Natural Science Foundation of China[U1913210] ; Beijing Natural Science Foundation[M22008] ; Youth Innovation Promotion Association of ChineseAcademy of Sciences (CAS)[2020140] ; Strategic Priority Research Program of CAS[XDB32040000] |
| Funding Organization | National Natural Science Foundation of China ; Beijing Natural Science Foundation ; Youth Innovation Promotion Association of ChineseAcademy of Sciences (CAS) ; Strategic Priority Research Program of CAS |
| WOS Research Area | Robotics |
| WOS Subject | Robotics |
| WOS ID | WOS:000889895900001 |
| Publisher | CAMBRIDGE UNIV PRESS |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | http://ir.ia.ac.cn/handle/173211/50798 |
| Collection | 复杂系统管理与控制国家重点实验室_先进机器人 |
| Corresponding Author | Zhou, Xiao-Hu; Xie, Xiao-Liang |
| Affiliation | 1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 3.China Univ Min & Technol, Sch Mech Elect & Informat Engn, Beijing, Peoples R China 4.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, Beijing, Peoples R China |
| First Author Affilication | Institute of Automation, Chinese Academy of Sciences |
| Corresponding Author Affilication | Institute of Automation, Chinese Academy of Sciences |
| Recommended Citation GB/T 7714 | Li, Hao,Zhou, Xiao-Hu,Xie, Xiao-Liang,et al. Discrete soft actor-critic with auto-encoder on vascular robotic system[J]. ROBOTICA,2022:12. |
| APA | Li, Hao.,Zhou, Xiao-Hu.,Xie, Xiao-Liang.,Liu, Shi-Qi.,Gui, Mei-Jiang.,...&Hou, Zeng-Guang.(2022).Discrete soft actor-critic with auto-encoder on vascular robotic system.ROBOTICA,12. |
| MLA | Li, Hao,et al."Discrete soft actor-critic with auto-encoder on vascular robotic system".ROBOTICA (2022):12. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment