中国科学院自动化研究所机构知识库

对于工业场景中作业目标的位姿测量是工业自动化生产过程的关键环节。在工业场景下，传统的位姿测量方法往往依赖于人工安装在目标表面的靶标、定位标志等装置，需要人工介入，而且测量装置与作业目标存在接触，致使测量效率相对较低、自动化程度不足，难以满足当前工业自动化领域的需求。

立体视觉测量技术采用立体视觉传感器对测量目标进行数据采集，结合自动化的数据处理，可以实现自动化、无接触的位姿测量，具有信息量大、效率较高等特点，相比传统方法具有一定优势。然而，在工业场景中，由于光照条件较差、噪声干扰严重、测量目标纹理简单、目标与背景的对比度较弱，传统的视觉测量方法往往难以在工业场景下准确、稳定地应用。此外，实际的工业自动化生产任务在位姿测量系统的测量精度、测量范围、系统稳定性、部署效率和硬件成本等方面有着较高的要求，适用于一般场景的视觉测量系统往往难以满足工业场景的应用需求。因此，基于立体视觉的工业场景目标位姿测量问题具有重要的研究意义和工程价值。本文基于立体视觉测量技术，针对不同工业场景下的目标位姿测量问题展开研究，论文的主要工作如下：
1. 针对各类工业场景目标位姿测量任务中不同的精度和测量范围的需要，设计并搭建了面向工业场景的立体视觉传感系统，包括基于RGB-D视觉的全局视觉传感系统和基于线结构光视觉手眼系统的局部视觉传感系统。针对线结构光视觉手眼系统中，相机参数、结构光平面参数及手眼参数标定流程繁琐、装置复杂、效率较低的问题，提出了一种借助平面标定板的线结构光视觉手眼系统参数同时标定方法，标定过程高效、装置简单，可以满足工业场景的应用要求。
2. 针对工业场景下目标位姿估计任务中，图像数据量较小、工业场景环境杂乱、光照条件较差、测量范围较大、测量距离较远等问题，采用RGB-D立体视觉传感器采集场景图像，提出了一种适用于复杂工业场景的深度神经网络进行目标分割，该网络采用跳跃连接和共享池化因子等结构，可以在有限的训练数据下完成精确的分割，根据分割所得的目标点云，提出了一种结合目标朝向识别网络和多候选模型配准的位姿估计方法，选择目标朝向对应的候选模型进行配准，防止算法陷入局部最优，进而实现目标位姿的稳定估计。
3. 针对工业场景下近距离目标位姿测量任务中，测量效率相对较低、鲁棒性不足、易受光学噪声影响等问题，采用装有滤光器件的线结构光视觉手眼系统采集目标点云，缓解光学噪声的影响，同时，采用特定视角采样的模型点云作为位姿测量的参考模型，减少了位姿配准过程中的离群点，提升了配准的鲁棒性。在测量阶段，提出了一种基于点云主轴的粗配准方法，该方法不涉及逐点的特征计算，运算效率较高且不易受噪声影响，可以为后续的ICP精配准提供准确的初始位姿，进而准确、高效、稳定地完成目标位姿的测量。

4. 针对工业场景下大型目标位姿测量中，传感器的测量范围和精度往往相互制约，并且因无法自动识别目标特征而依赖于人工安装的标志物进行测量，造成的测量效率较低、精度不足、自动化程度有限的问题，采用由全局视觉和局部视觉组成的立体视觉位姿测量系统进行位姿测量。根据全局视觉系统采集的大范围数据，提出一种基于模型拟合的方法对目标进行分割和粗定位，并结合粗定位结果，采用局部视觉系统采集目标局部点云数据，通过降维的方式转换为二维灰度图像进行关键点检测和定位，并映射回三维空间完成位姿的精确计算，进而完成对大型目标位姿的大范围、高精度、自动化测量，满足工业场景中面向大型目标作业的应用要求。

最后，对本文的研究工作进行了总结，并提出了进一步的研究计划。

Pose measurement of object in industrial scenario is a crucial procedure for industrial automation applications. In industrial scenario, the traditional pose measurement methods still rely on fiducial targets or markers that need to be manually installed on the surface of the object. These methods require human intervention and involve contact with the measured object, resulting in low efficiency and low automation level, inadequate to meet application requirements in industrial scenario.

3D vision measurement technology adopts 3D vision sensor to collect data from the measured object. Combined with automatic data processing, it can realize automatic and non-contact pose measurement. 3D vision technology is capable of acquiring large quantity of information with high efficiency, outperforming the traditional methods. Nevertheless, in industrial scenario, due to the unstable illumination, severe noise interference, and poor conspicuousness of object feature, it is difficult for the traditional vision measurement methods to accurately and stably function. In addition, there are relatively high standards in measurement precision, measurement range, system stability, deploying efficiency and hardware cost for the pose measurement system in industrial automation assignments. Hence the vision measurement systems for ordinary scenarios can hardly meet these requirement. Therefore, it is of great research significance and engineering value to study object pose measurement problems based on 3D vision in industrial scenario. This thesis presents several pose measurement researches conducted in different industrial scenarios, based on 3D vision technology. The major contributions of this thesis are listed as follows:

1. To satisfy the various demands of measurement precision and range in different industrial scenarios, 3D vision system designed towards industrial scenario applications is established. The system includes global vision system based on RGB-D vision sensor, and local vision system based on robotic eye-in-hand system with line structured light vision sensor. For a line structured light vision sensor in robotic eye-in-hand system, the traditional calibration process of camera parameters, structured light parameters and hand-eye parameters is rather intricate and inefficient, using complex calibration targets as auxiliary. To address this, a simultaneous calibration method for these parameters using a simple planar calibration board is proposed to realize accurate calibration with high efficiency and low cost, meeting the requirement of industrial applications.       
2. For object pose estimation in industrial scenario, the general problems that cause difficulty to the measurement include the insufficiency of image data, the cluttered industrial environment, the poor illumination, the large measurement range and long measurement distance. In order to solve these issues, RGB-D vision sensor is adopted to capture the image of the scene. A deep neural network aimed for industrial scenario is proposed to segment the object from the scene. The network possesses skip connection structure and adopts shared pooling indices, capable of achieving accurate segmentation using limited training data. The segmented object point cloud is then fed to orientation recognition network to determine the basic orientation, followed by multi-candidate model registration using the model candidate corresponding to its orientation to robustly accomplish the pose estimation process.      
3. For the close-range object pose measurement in industrial scenario, there exist various issues including low measurement efficiency, inadequate robustness, and poor resistance to the optical noises. To address these issues, robotic hand-eye measurement system with line structured light vision sensor equipped with optical filters is adopted to alleviate optical noises from the input. Additionally, partial model point cloud sampled from specific viewpoint is adopted as the canonical model to diminish the outliers during registration. During on-site measurement, a coarse alignment method based on principal axes is proposed. This method does not involve per-point feature computation, and exhibits high efficiency and high robustness under noises during measurement. The proposed coarse alignment method provides accurate initial pose estimation for the subsequent ICP fine registration, making the whole object pose measurement process precise, efficient and stable.    
4. For the pose measurement of large scale object in industrial scenario, the major drawbacks of current measurement methods include the mutual constraint between the measurement range and precision of the sensor, the inability to automatically recognize the object feature, and the reliance of the manually installed fiducial targets, which make the measurement inefficient, inaccurate and less automatic. To avoid these issues, a 3D vision measurement system composed of global vision and local vision system is established. Based on the large range image data collected by the global vision system, a model fitting method is proposed for the object segmentation and localization. According to the object localization result, the local vision system is used to capture the local point cloud data of the large scale object. Then dimensionality reduction is conducted upon the resulting local point cloud to convert the 3D point cloud to 2D greyscale image, on which the keypoints are detected. The obtained 2D keypoints are thereafter mapped back to 3D space for precise pose computation. Thereby, the automatic pose measurement is accomplished on large scale object with large measuring range and high precision, satisfying the application requirements in industrial scenario.

Finally, this thesis is summarized and the further research plans are suggested.