中国科学院自动化研究所机构知识库

无人机具有的灵活性大、成本低、效率高且安全性高等一系列优点使之成为了获取低空航拍影像的重要工具之一。无人机现在被广泛应用于城市规划、森林火灾监控和定位以及农业监测等诸多领域。一个高质量的实时正射拼图可以更好的帮助完成诸如灾情风险控制和救援等任务。由于机载无人机有限的观测视野和相对低的飞行高度，利用无人机航拍图像实时生成正射图逐渐变成一种十分必要的方案。考虑到正射图拼接涉及到大量的计算，实现高效准确的航拍图像拼接是一个亟待解决的问题。

基于以上分析，本文重点关注融合多源信息尤其是GPS信息的航拍场景实时正射图的生成与拼接。研究内容分为两大部分：第一部分是根据无人机航拍中的视觉信息和相关的地理信息如GPS信息实时地估计出准确的无人机位姿和稀疏点云，第二部分是根据第一部分估计出的位姿、点云以及图像本身实时地得到一个高质量的正射拼图。

本文的主要创新点如下：

1. 提出了一种无人机位姿的优化方法，将GPS信息与基于关键帧的单目SLAM算法深度融合，设计了基于地理信息的位姿估计算法，可以处理纯视觉方法中的误差累积问题，增强了拼图过程对GPS误差变化的鲁棒性；
2. 提出了一种局部正射图生成与拼接方法，针对平面和非平面场景的不同特性，利用相机姿态和像素中心性指标实现注重正射性的图像选择，显著提升了拼接图像的正射性；
3. 构建了一个融合地理视觉信息的场景正射影像实时生成系统。使用了基于关键帧的单目SLAM算法和图像拼接技术。该系统在无人机航拍的实际场景应用效果良好。

总之，基于本文提出的方法，通过在现有大量数据集上的测试、消融实验以及对比实验证明了该算法的准确性、实时性和鲁棒性并基于此设计了一个完整的正射图拼接系统，该系统可以在不同场景下进行高效准确的航拍图像拼接。

Unmanned aerial vehicles (UAV) have a series of advantages, such as high flexibility, low cost, high efficiency, and safety, which make them an important tool for obtaining low-altitude aerial photography. Drones are now widely used in many fields, including urban planning, forest fire monitoring and positioning, and agricultural monitoring. A high-quality real-time digital orthophoto map can better help complete tasks such as disaster risk control and rescue. As the observation field of view of onboard drones is limited and the flying height is relatively low, using drone aerial images to generate orthophotos in real-time has gradually become a necessary solution. Considering that digital orthophoto map involves a large amount of calculation, achieving efficient and accurate aerial image stitching is an urgent problem to be solved.

Based on the above analysis, this thesis focuses on the generation and stitching of real-time orthophotos in aerial photography scenes by fusing multiple sources of information, especially GPS information. The research content can be divided into two main parts: the first part is to estimate the accurate pose of the unmanned aerial vehicle and sparse point cloud in real-time based on the visual information from the UAV aerial photography and related geographic information such as GPS information; the second part is to obtain a high-quality orthoimage in real-time based on the estimated pose, point cloud, and the image itself.

The main innovations of this thesis are as follows:

1. A method for optimizing the pose of UAVs is proposed. The GPS information is deeply integrated with the monocular SLAM algorithm based on keyframes, and a pose estimation algorithm based on geographic information is designed, which can deal with the error accumulation problem in the pure vision method and Enhance the robustness of the mosaic process to changes in GPS errors;
2. A local orthomap generation and stitching method is proposed. According to the different characteristics of planar and non-planar scenes, the camera pose and pixel centrality index are used to process the pixel selection focusing on orthogonality, which significantly improves the orthogonality of stitched images;
3. A real-time generation system of scene orthophotos integrated with geographic and visual information is constructed. The keyframe-based monocular SLAM algorithm and image stitching technology are used. The system works well in the actual scene of UAV aerial photography.

In conclusion, based on our method proposed in this thesis, the accuracy, real-time performance, and robustness have been demonstrated through testing, ablation experiments, and comparative experiments on existing large datasets. Based on this, a complete orthophoto stitching system has been designed, which can efficiently and accurately stitch aerial images in different scenes.