An End-to-End Deep Reinforcement Learning Based Modular Task Allocation Framework for Autonomous Mobile Systems

doi:10.1109/TASE.2024.3367237

	An End-to-End Deep Reinforcement Learning Based Modular Task Allocation Framework for Autonomous Mobile Systems
	Ma, Song 1; Ruan, Jingqing 2; Du, Yali 3; Bucknall, Richard 1; Liu, Yuanchang 1
发表期刊	IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
ISSN	1545-5955
	2024-02-21
页码	15
通讯作者	Liu, Yuanchang(yuanchang.liu@ucl.ac.uk)
摘要	Intelligent decision-making systems that can solve task allocation problems are critical for multi-robot systems to conduct industrial applications in a collaborative and automated way, such as warehouse inspection using mobile robots, hydrographic surveying using unmanned surface vehicles, etc. This paper, therefore, aims to address the task allocation problem for multi-agent autonomous mobile systems to autonomously and intelligently allocate multiple tasks to a fleet of robots. Such a problem is normally regarded as an independent decision-making process decoupled from the following task planning for the member robots. To avoid the sub-optimal allocation caused by the decoupling, an end-to-end task allocation framework is proposed to tackle this combinatorial optimisation problem while taking the succeeding task planning into account during the optimisation process. The problem is formulated as a special variant of the multi-depot multiple travelling salesmen problem (mTSP). The proposed end-to-end task allocation framework employs deep reinforcement learning methods to replace the handcrafted heuristics used in previous works. The proposed framework features a modular design of the reinforcement learning agent which can be customised for various applications. Moreover, a real-robot implementation setup based on the Robot Operating System 2 is presented to fulfil the simulation-to-reality gap. A warehouse inspection mission is executed to validate the training outcome of the proposed framework. The framework has been cross-validated via both simulated and real-robot tests with various parameter settings, where adaptability and performance are well demonstrated. Note to Practitioners-This paper is motivated by the problem of dispatching a fleet of autonomous mobile robots to tackle a mission that can be resolved into multiple waypoint-following tasks. An end-to-end modular framework is proposed, making task allocation decisions based on the given waypoint information. By using the reinforcement learning technique, the deep neural network could learn sophisticated policies for allocating tasks. The policies are trained in a specific pattern which ensures their joint optimisation for a solver that outputs the near optimal task execution sequences in an efficient way. This leads to a multiple travelling salesmen problem (mTSP) solution. Pre-trained policies are tested in several industrial scenarios reflecting the applications of search and rescue, maritime surveying, and warehouse automation, among others. A hardware implementation configuration based on the Robot Operating System 2 is also presented to support the practical deployment the framework.
关键词	Deep reinforcement learning task allocation multi-agent planning field robotics
DOI	10.1109/TASE.2024.3367237
关键词[WOS]	TRAVELING SALESMAN PROBLEM ; OPTIMIZATION ; ALGORITHMS ; PSO
收录类别	SCI
语种	英语
资助项目	Dean's Prize of University College London Faculty of Engineering Sciences
项目资助者	Dean's Prize of University College London Faculty of Engineering Sciences
WOS研究方向	Automation & Control Systems
WOS类目	Automation & Control Systems
WOS记录号	WOS:001179660600001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/56943
专题	复杂系统认知与决策实验室_听觉模型与认知计算
通讯作者	Liu, Yuanchang
作者单位	1.UCL, Dept Mech Engn, London WC1E 7JE, England 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 3.Kings Coll London, Dept Informat, London WC2R 2LS, England
推荐引用方式 GB/T 7714	Ma, Song,Ruan, Jingqing,Du, Yali,et al. An End-to-End Deep Reinforcement Learning Based Modular Task Allocation Framework for Autonomous Mobile Systems[J]. IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING,2024:15.
APA	Ma, Song,Ruan, Jingqing,Du, Yali,Bucknall, Richard,&Liu, Yuanchang.(2024).An End-to-End Deep Reinforcement Learning Based Modular Task Allocation Framework for Autonomous Mobile Systems.IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING,15.
MLA	Ma, Song,et al."An End-to-End Deep Reinforcement Learning Based Modular Task Allocation Framework for Autonomous Mobile Systems".IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (2024):15.