毕业生知识库

随着计算机视觉和机器人技术的发展以及科学研究和生产生活的需要,多旋翼无人机在农业植保、抢险救援、运动航拍、空间探测、工业制造等领域的应用越来越广泛。无人机系统为完成位置姿态控制、环境感知、轨迹和任务规划等依赖于鲁棒精确的定位建图,特别是在空中复杂条件下,如何实现无人机的鲁棒精确快速的定位建图日益受到重视。基于多信息融合的无人机定位建图能够有效的提升无人机在多种条件下定位建图的鲁棒性和精度,利用相机、IMU(Inertial MeasurementUnit) 等传感器能够有效的降低无人机的载重,提升续航能力,为无人机的稳定飞行提供鲁棒实时的定位信息,是无人机开展其他工作的基础和前提。本文针对无人机的定位建图以及视觉标定等问题展开研究,主要的工作和贡献有:

1. 针对离焦图像提出了一种基于脊不变性的鲁棒标定方法。对于离焦图像,传统的标定方法由于图像的模糊导致提取的控制点位置不准确,导致标定精度下降,本文利用脊线提取代替传统标定方法中的边缘提取,精确的提取图像的控制点,完成标定。本文给出了离焦图像中脊线的不变性条件,利用基于带状线条的标定模式,可以避免反卷积求解控制点。然后分析了脊线的显著性与图像的离焦量之间的关系。为了提升脊线的显著性,便于精确的提取脊线,利用梯度比率方法估计图像离焦量,对图像进行二次模糊,使得图像中的脊获得最大显著性,精确获取标定图像的控制点,实现了采用离焦图像对相机的精确标定。

2. 提出了一种结合点线特征及结构信息的视觉惯性里程计方法。在城市环境或者室内环境中,存在丰富的线段信息,利用这些线段特征可以作为点特征的补充,提升定位的精度和鲁棒性。本文首先分析了利用 Plucker 坐标以及Plucker 矩阵进行位姿估计以及光束平差优化的方法,然后结合点线特征及结构信息在滑动窗口中进行局部光束平差优化。利用线段检测器检测图像中的线特征,同时提取图像中的点特征,根据点特征的差异度判别特征是否适合被跟踪,选择合适的特征用于跟踪。利用 J-linkage 对提取的线段进行分类,计算图像的消失点,利用消失点对应主方向作为结构信息添加至优化过程。为了降低计算资源消耗,使用基于滑动窗口的稀疏性光束平差法对里程计的估计进行优化。

3. 提出了一种结合语义信息的视觉里程计方法。利用图像中的平面信息,结合平面中点特征的单应约束,有助于提升定位的精度。本文首先利用全卷积网络实现图像中平面分割,然后利用稀疏特征完成相机的位姿跟踪,对稀疏特征使用三角化实现点的三维重建。利用三角化重建的三维点以及图像分割结果拟合局部地图中的分段平面。拟合的分段平面结构构成额外的单应约束,在优化中利用图优化方法完成非线性最小二乘,从而提升了视觉定位的精

4. 提出了一种融合全卷积网络深度图预测及几何深度值估计的方法。视觉几何法估计的深度和深度网络预测的深度各有优缺点,可以融合得到更加精确稠密的深度图。利用校准的双目图像训练深度图预测网络,通过最小化每个像素重投影的光度残差,构成无监督的全卷积网络深度预测,利用双目数据的左右图像重建避免深度预测网络对于深度真值的依赖。然后将深度预测结果和多视角计算的深度进行融合,获取更加精确稠密的深度图,利用TSDF(Truncated Signed Distance Fields) 及光线透射法完成稠密地图重建。另外为了降低视觉里程计在长时间运行中产生的漂移,利用离线建立的地图,基于词袋模型构建视觉定位系统,相对于地图的定位方式不具有累计漂移的特点。在离线建图中,通过闭环检测及位姿图优化,得到闭环修正的轨迹及地图点, 利用全局 BA(Bundle Adjustment) 对轨迹和地图点进行优化。在重定位中,通过加载地图,利用 BOW 搜索,完成重定位。

With the development of computer vision and robotics, as well as the requirement from industrial production and people’s daily life, multi-rotor UAVs (Unmanned Aerial Vehicles) have been more and more widely used. It can act as a significant role in agricultural plant’s protection, rescue, aerial photography, space exploration and industrial manufacturing. UAV system generally relies on robust and accurate localization and mapping to complete position and attitude control, environment perception, trajectory and mission planning. Specially how to realize the robust and accurate localization of UAVs under aerial complex conditions becomes an important research topic. Based on multi-information fusion, UAV’s localization and mapping can be effectively improved with the robustness and accuracy under a variety of conditions. Utilizing sensors such as camera, IMU can effectively reduce the carrying capacity of UAVs and enhance flight endurance. The system to provide robust and real-time location information can be an important and fundamental component of stable running UAV. This paper focuses on the localization of UAV and system calibration. The main work and contributions are as follows:

1. For defocused images, a robust calibration method based on broad brush line calibration pattern is proposed. Due to the image blur, the traditional calibration method is inaccurate caused by the position of extracted control points is inaccurate, resulting in a decrease in calibration accuracy. We first demonstrate the defocused image ridge invariant condition, i.e. the position of the extrema is invariant to the defocus amount of the image. We also analyse the saliency of the ridge with respect to the defocus amount in the calibration image. After estimating the image defocus amount, the convolutional kernel is determined. The original image is reblurred with the kernel, and the maximum saliency of the image ridge can be obtained. The precise position control points of the calibration image are computed by solving the intersection. The proposed method gives better precision for defocused image with respect to traditional method.

2. A visual inertial odometry system is proposed which combines point, line features and structure information. In the urban environment or indoor environment, there is rich line segment information, these line segment features can be used as a complement to point features to improve the accuracy and robustness of the visual inertial odometry. We first demonstrate the pose estimation and bundle adjustment optimization with lines by using Plucker coordinates and the Plucker matrix. A line segment detector is used to detect the line features in the image, and point features in the image are extracted at the same time, and features suitable for tracking are selected. J-linkage is used to classify the extracted line segments, calculate the vanishing point of the image, as structural information which are added in the optimization. In order to reduce the computational resources, a sparse bundle adjustment is performed on a sliding window.

3. A visual odometry combined with semantic information is proposed. Using the plane information in the image and combining the homography constraints of the point features in the plane helps to improve the accuracy of localization. We first use the fully convolutional network to segment the plane in the image, use the sparse features to complete the camera pose tracking. By triangulating the sparse features to reconstruct of the 3D points. The points on plane are fitted to get the piecewise plane in local map. The piecewise plane can form homography constraint in the optimization, and enhance the precision of the pose estimation of visual odometry.

4. A depth estimation method is proposed by combining learning based depth map prediction and geometric depth estimation. The depth estimated by learning based network and the depth from visual geometry method have their own advantages and disadvantages, and can be fused to obtain a more accurate and dense depth map. Using rectified stereo images to train the fully convolutional depth map prediction network, an unsupervised full convolution network is constructed by minimizing the photometric residuals of each pixel reprojection, in which the depth ground truth is free. The depth prediction results and the depth of the visual geometry method are fused to obtain a more accurate and dense depth map, then we use grouped raycasting method to integrate the depth map and obtain the truncated signed distance field(TSDF) map. In addition, in order to reduce the drift of visual odometry during long-term running, offline visual mapping based on bag of word(BOW) model is used to complete relolicalization. In the offline mapping, the loop is detected to correct the trajectory with global bundle adjustment to optimize the trajectory and map points. By loading the offline built map, relocalization is completed with BOW search to decrease the drift of the odometry.