中国科学院自动化研究所机构知识库

        近年来，随着计算机视觉技术的进步，基于图像的广域多目标探测技术在智能交通、公共安全、体育转播以及军事态势感知等方面取得了巨大成功。然而，传统的探测技术在应对复杂场景、大面积观测视野和动态目标分布等实际工作环境时，在观测视野、搜索效率与检测速度等方面均存在明显不足。在未知广域环境中，如何提升目标搜索与感知能力，并快速准确地获得感兴趣目标的高清图像，值得深入研究。因此，本文针对广域场景多目标探测中的关键问题，从视野快速扩展、广域场景建模和目标快速搜索与感知三个方面逐层递进地展开深入研究。论文的主要研究内容如下：

        （1）针对广域目标难以探测的问题，提出了一种基于机械粒子滤波的广域多目标主动视觉探测方法，实现了广域场景中300 帧/秒的探测速度。在大视野场景下，感兴趣目标的图像分辨率较低且目标分布无规律，使得现有探测方法难以精确探测。为了同时实现大视野覆盖和目标快速探测，本文提出了一种大视野多目标快速探测方法。首先，控制光学扫描振镜配合高速视觉平台实现高速视场切换，获得大视野实时扩展能力。其次，为了提高目标探测效率，提出了基于机械粒子滤波的目标探测框架来构建广域场景中目标的概率分布模型，并迭代定位出感兴趣目标在大视野场景中的位置。此外，为了减少粒子扫描开销，引入了基于中心粒子的扫描路径规划算法，通过将所有粒子按概率和距离划分成多个粒子群来减少路径求解与扫描耗时。最后，多目标探测实验结果表明，该方法能够快速准确地探测广域场景中的感兴趣目标。

        （2）针对广域目标搜索效率低的问题，提出了区域概率图指导的快速广域多目标探测方法，显著提升了高概率区域目标的探测性能。现有的广域探测方法多采用全局建模方式，使得探测初期大量采样子图像包含无效信息，导致探测收敛速度变慢。首先，为提升探测效率，本文提出了区域概率图生成方法。该方法利用语义分割模型对全景图像进行分割，通过联立目标-区域语义关系和分割出的场景结构信息，构建出区域概率测量函数，进而生成区域概率图。随后，提出了区域概率图指导的粒子滤波方法，该方法通过按概率值分配不同比例的粒子到不同区域，使采样初期的粒子能够集中分布在高概率区域。评估与泛化性实验结果表明，所提方法既能够对场景进行有效的概率建模，还具有优异的探测精度和很好的实时性。该方法最大可以提升38.38% 的目标检测效率。

        （3）针对广域目标探测任务中，高速目标检测算法定位精度差的问题，设计了一种多帧信息融合的硬件高速目标检测方法，实现了10,000 帧/秒的高速目标定位。为解决已有的基于HOG 特征的高速硬件目标检测算法因检测窗口步长过大导致定位误差大的问题，提出多帧图像目标信息融合技术。首先，制定了检测区域自适应调整策略，有效减少原始检测窗特征提取的硬件资源消耗。其次，提出基于多帧信息同步的数据融合生成方法，显著降低了多帧信息融合的目标位置误差。最终，基于FPGA 硬件平台实现了本文提出的高速目标检测算法，该系统能够同时获得目标的位置与图像数据。实验结果表明，所设计的方法能够在保持高速目标检测性能的同时有效降低定位误差。

        （4）针对广域目标感知速度慢的问题，构建了一套广域多目标实时探测系统，实现了复杂广域环境下的动态目标高速探测。与仿真和室内场景相比，室外环境存在着复杂的目标运动和场景干扰，为了实现复杂动态场景下多目标的快速探测，首先，控制两轴振镜进行高速二维视野扩展，然后基于大视野广角相机的区域概率指导，引导高速长焦相机进行高效目标探测，实现了广域高速探测系统。为进一步提升系统的实时性，设计了一种基于视觉反馈的高并行度控制策略，通过将所提广域探测算法集成到高并行度、低延迟的控制范式中，实现了探测流程加速。最后，一系列复杂场景实验验证了所构建的广域多目标探测系统的有效性和鲁棒性。

        In recent years, with the rapid development of computer vision algorithms, object detection-based wide-area video surveillance technology has made tremendous success in the fields of intelligent transportation, public security, sports broadcasting, military situational awareness, and more. However, when dealing with real-world environments like complex scenes, large monitoring Field-of-Views (FOVs), and dynamically distributed objects, the performance of traditional surveillance technology has major deficiencies in terms of searching efficiency, detection speed, imaging quality, covering FOV, etc. How to achieve fast and accurate object searching and detection abilities while capturing high-quality images of objects of interest requires further in-depth study. Consequently, aiming at the key issues in wide-area multiobject detection, this dissertation starts from the aspects of perceptual FOV expansion, wide-area scene modeling, and fast object detection, which conducts several studies in a coarse-grained to fine-grained manner. The main work and contributions of this dissertation are as follows.

        (1) A mechanical particle filter-based active vision system for wide-area multiobject detection is proposed to deal with object detection problems in a large scene, which achieved 300 fps detection speed. In the scenario of a wide area, the low target image resolution and irregular distribution of objects of interest make existing detection algorithms poor in search efficiency and lower localization accuracy. To simultaneously achieve a large covering FOV and high-speed detection performance, a large FOV multiobject detection algorithm is proposed. Firstly, an optical scanning galvano-mirror and a high-speed vision platform are utilized to provide fast FOV switching, which achieves a wide-area covering ability. Secondly, to improve the detection efficiency, a mechanical particle filter framework is proposed to construct the objects' probability distribution model and iteratively locate the target object's position in the large FOV, where a single particle is represented by an image that is captured by the galvano-mirror. Finally, a center-partitioned scanning path planning algorithm is introduced to reduce the total solving and scanning costs, where all the particles are partitioned into several particle sets according to their weights and distances to the center particles. The experimental results show that the proposed methods could detect desired objects distributed in a wide area quickly and accurately.

        (2) A region probability map-guided fast wide-area multiobject detection method is proposed for the problem of low object searching efficiency in a wide area. Since many wide-area detection methods are modeling the distribution of the whole large scene, resulting in many sampling images containing invalid information during the early stages of searching, they cannot provide effective guidance for the next stage. To improve searching efficiency, this dissertation first proposes a region probability map generation algorithm to process the panoramic image. This algorithm employs a semantic segmentation model to segment the panoramic image that corresponds to the wide-area scene. By considering the prior statistical information of object-region semantic relationships and the scene structure, a region probability measurement function is built. Subsequently, when applying this function to the segmentation mask, each region is assigned a probability measurement that indicates how likely the desired object will occur in that region. Finally, a region probability map-guided particle filter method is proposed to proportionally assign different numbers of particles to different regions according to the relative probabilities, which leads to the sampling particles being concentrated more on high-probability regions at the early stage. The results of the evaluation and generalization experiments show that the proposed detection method could efficiently model the probability of the whole scenario while achieving superior real-time performance, higher accuracy, and fast detection speed. The method can improve the target detection efficiency by a maximum of 38.38%.

        (3) To compensate for the high-speed object detection deviation, an FPGA-based ultrahigh-speed object detection method with multi-frame information fusion is proposed, which achieves 10,000 fps target localization. Owing to the localization deviation caused by the large stride between detection windows, the Histogram of Oriented Gradient (HOG) feature-based high-speed detection algorithm is limited in localization accuracy. This dissertation introduces an image data and synchronization signals module. Firstly, a starting point of detection area adjustable strategy is designed to reduce the fixed stride in the original detection window. Subsequently, to accurately represent the start point of the detection window, a synchronization signals module is proposed for real-time detection area synchronization. Based on these proposed modules, once the detector detects objects of interest, the post-process module could simultaneously get the position of starting point and the image data. Then, the algorithm only stores the maximum HOG feature and its corresponding coordinates at the bottom of the image frame. Eventually, the proposed method is implemented on the FPGA-based high-speed vision platform. Experiments show that the designed algorithm could achieve ultrahigh speed while reducing localization deviations.

        (4) Aiming for the problem of slow perception of wide-area FOV, a real-time wide-area multiobject detection system is constructed by utilizing a 2-axis galvano-mirror for high-speed FOV expansion. Compared with simulation and indoor environments, the outdoor scenarios are full of object movements and environmental noises. To verify the effectiveness and robustness of the above proposed wide-area multiobject detection algorithms, an active detection system that is based on a panoramic camera, a high-speed telephoto camera, and an ultrafast switching galvano-mirror is implemented. Specifically, the principle of rapid FOV expansion for galvano-mirror is analyzed, and the information processing relationship between each module of the detection system is briefly described. Then, to improve the system's real-time performance, a visual feedback-based high-parallelism control method is proposed. The method integrates the region probability map generation module and the particle filter detection method of the wide-area detection algorithm into a high-parallelism, low-latency control paradigm, resulting in a detection acceleration. Eventually, by using the constructed wide-area multiobject detection system, extensive experiments are conducted on the complex environment to verify the effectiveness and robustness of the system.