CASIA OpenIR
基于深度学习的自主空中加油目标检测与跟踪研究
孙思洋
Subtype博士
Thesis Advisor徐德
2020-06
Degree Grantor中国科学院大学
Place of Conferral北京市石景山区玉泉路19号
Degree Discipline模式识别与智能系统
Keyword深度学习 强化学习 目标检测 目标跟踪 关键点检测 单目视觉测量 自主对接控制 空中加油
Abstract

近年来,随着基于视觉的目标检测和跟踪技术的快速发展,基于视觉的智能系统在各个领域都有着广泛的应用。在航空航天领域,基于视觉的自动控制系统也日益受到重视,其中,基于视觉的空中加油自主对接系统对我国军事能力的发展具有重要意义。在空中复杂条件下,如何快速准确地实现对锥套目标的检测、跟踪以及位置测量,是空中加油自主对接系统亟待解决的问题。本文以软管式空中加油为背景,研究基于深度学习的目标检测、跟踪以及基于视觉的目标位置测量方法。本文的主要工作和贡献如下:

1)针对单阶段目标检测网络检测精度较低的问题,提出了一种基于多感受野和弱监督分割的目标检测网络。该网络由两个分支构成:基于多感受野模块的检测分支和关注小目标的弱监督分割分支。在检测分支中,利用不同尺度的特征构造多尺度目标检测预测层。同时,为了提高目标检测的精度,在多尺度目标检测预测层上增加了多感受野模块,以不同的权重关注物体及其相邻背景的不同空间位置。在弱监督分割分支中,设计了关注小目标的弱监督分割模块,该模块将小目标的分割任务作为目标检测的辅助任务,可以提高小目标的检测精度。该网络采用多任务训练的方式,同时对检测分支和分割分支进行端到端的有监督训练。分别在空中加油锥套目标检测数据集和通用目标检测数据集上,验证了该网络的有效性。

2)针对图像变化导致的检测难题,提出了一种基于深度学习和强化学习的空中加油锥套目标检测和跟踪方法。在空中复杂条件(如强光干扰、气流扰动、目标遮挡、目标尺度变化及姿态变化等)下,空中加油锥套目标图像差异明显,导致目标检测及跟踪难度大。为解决上述难题,提出了一种融合目标检测和跟踪的方法。其中,目标检测器是基于YOLO框架进行设计的,能够快速检测锥套目标,获得目标的粗略位置;目标跟踪器采用基于强化学习的方法对目标检测器得到的目标框进行调整,提高跟踪框与真实目标框的交并比,进而提高目标跟踪的精度。利用搭建的空中加油地面模拟平台,对上述方法进行了验证。

3)针对锥套目标的实时三维测量问题,提出了一种基于空中加油目标模型的单目视觉测量方法。该方法包含两部分:基于多任务并行的深度卷积神经网络的锥套目标关键点检测和基于双椭圆的空中加油锥套目标位置测量。空中加油锥套目标由内部黑心部分、伞骨和外部伞套部分构成。根据空中加油锥套目标的几何特点,关键点检测模型检测锥套内部黑心部分的16个关键点和锥套外部伞套部分的32个关键点,将锥套目标关键点检测结果作为视觉测量的图像特征。同时,由于空中加油锥套目标内部黑心部分和外部伞套部分的投影在图像平面上是圆形或是椭圆,结合空中加油锥套目标关键点检测结果,设计了一种基于双椭圆的空中加油锥套目标位置测量模型,提高了空中加油目标位置测量的准确性。

4)搭建了双机器人空中加油自主对接实验验证平台,实现了对视觉检测、跟踪以及测量方法的有效验证。该平台由两台机器人和一台嵌入式设备组成。其中,两台机器人分别用来模拟空中加油自主对接过程中的加油机和受油机。嵌入式设备Nvidia Jetson TX2作为计算平台,将所提出的空中加油锥套目标检测、跟踪、位置测量和视觉控制算法部署在该嵌入式设备上。实验结果验证了上述方法的有效性。

Other Abstract

In recent years, with the rapid development of vision-based object detection and object tracking technologies, vision-based intelligent systems are widely used in various fields. In the field of aerospace, the vision-based autonomous control system is getting more and more attention. The vision-based autonomous docking system of aerial refueling has important significance for the development of our country’s military capability. How to quickly and accurately realize drogue object detection, tracking and position measurement under the complex aerial condition, is the main problem to be solved for the autonomous docking system of aerial refueling. In this dissertation, the object detection and tracking based on deep learning and position measurement based on vision are studied for the probe-and-drogue aerial refueling. The main works and contributions of this dissertation are as follows.

(1) An object detection network based on multiple receptive fields and weakly-supervised segmentation is proposed to improve the accuracy of object detection for one-stage object detection network. The network consists of two parts: a detection branch based on multiple receptive fields module and a small-object-focusing weakly-supervised segmentation branch. In the detection branch, the multiple scale’s prediction layers of object detection are constructed using the features of different scales. At the same time, in order to improve the accuracy of object detection, the multiple receptive fields module is added in the prediction layer to focus on different spatial positions of the object and its adjacent background with different weights. In the weakly-supervised segmentation branch, this dissertation designs a small-object-focusing weakly-supervised segmentation module, which takes the segmentation of small object as the auxiliary task of object detection to improve the accuracy of small object detection. The network is trained in a multi-task’s manner with end-to-end supervised training for detection and segmentation branches. The effectiveness of the object detection network is verified on the drogue object detection dataset of aerial refueling and the general object detection dataset, respectively.

(2) A method of drogue object detection and object tracking of aerial refueling based on deep learning and reinforcement learning is proposed to overcome the difficulty of drogue object detection caused by image variations. Under the complex conditions of aerial environment (such as the interference of strong light, the disturbance of airflow, the occlusion of object, the scale’s change of object and the attitude’s change of object, etc), the image’s difference of drogue object for autonomous aerial refueling is obvious, which results in great problems to detect and track the drogue object. In order to solve the problems above, a method combining drogue object detection and object tracking is proposed in this dissertation. The object detector is designed based on YOLO architecture, which can quickly detect the drogue object and obtain the rough position of the object. The object tracker based on reinforcement learning adjusts object’s bounding box to improve the intersection over union of the tracking box and the ground truth, and then improves the accuracy of the object tracking. Finally, the proposed method for drogue object detection and object tracking of aerial refueling is verified on the ground simulation platform of aerial refueling.

(3) A monocular visual measurement method based on the model of the drogue object of aerial refueling is proposed for the real-time 3D measurement of drogue object. The method consists of two parts: a multi-task parallel deep convolution neural network for landmark detection of drogue object and a position measurement model based on double ellipses of drogue object for aerial refueling. The drogue object of aerial refueling is composed of the inner black center part, the umbrella bone and the outer umbrella cloth part. According to the geometric characteristics of the drogue object of aerial refueling, the landmark detection model is used for detecting 16 landmarks of the inner black center part of the drogue object and 32 landmarks of the outer umbrella cloth part of the drogue object. The result of drogue object’s landmark detection is used as the image feature for visual measurement. At the same time, because the projection of the inner black center part and the outer umbrella cloth part of the drogue object are circles or ellipses in the image plane, a double ellipse-based visual measurement model for aerial refueling is designed, which improves the accuracy of the position measurement of the drogue object for aerial refueling.

(4) An experimental verification platform based on two robots for autonomous docking process of aerial refueling is built, which is used to realize the effective verification of visual detection, tracking and measurement method. The platform is composed of two robots and an embedded device. The two robots are used for simulating the oil tanker and oil receiver of autonomous docking process of aerial refueling respectively. The embedded device Nvidia Jetson TX2 is used as a computing platform, and then the proposed drogue object detection and tracking of aerial refueling, position measurement method and visual control algorithm are deployed on the embedded device. The experimental results verify the effectiveness of the proposed methods.

Pages150
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/39059
Collection中国科学院自动化研究所
Recommended Citation
GB/T 7714
孙思洋. 基于深度学习的自主空中加油目标检测与跟踪研究[D]. 北京市石景山区玉泉路19号. 中国科学院大学,2020.
Files in This Item:
File Name/Size DocType Version Access License
博士论文-孙思洋20200308最终版明(19199KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[孙思洋]'s Articles
Baidu academic
Similar articles in Baidu academic
[孙思洋]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[孙思洋]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.