中国科学院自动化研究所机构知识库

光学元件是各种光学机构的关键性器件，其表面质量直接影响光学机构的性能。光学元件表面的缺陷通常会导致光路发生变化，造成光线散射，改变光线传输特性，轻则降低系统整体性能，重则损坏光学元件自身和光学机构，造成严重的财产损失甚至引发事故。在制造、加工和使用的过程中对光学元件表面进行缺陷检测是一项重要且必要的工作。使用基于机器视觉的检测方法，能够提高效率，降低成本，保护人的健康，不论对于工业生产还是科学研究都具有重要的现实意义。本文利用深度学习的方法，针对光学元件表面缺陷检测中的关键问题展开研究。

本文的主要贡献如下：

(1)针对光学元件表面微弱划痕的精确检测问题，提出了一种融合先验知识的划痕分割网络。通过对划痕在暗场成像的规律进行分析，得到了划痕在亮度、宽度和结构三个方面的特点，并依据三个特点设计了局部最大值指数(LMI)和方向敏感卷积算子(DSC)两个可计算的特征。作为划痕的先验知识的LMI和DSC可显著增强暗场图像中划痕与背景的差异。通过把LMI和DSC与一个编码—解码的分割网络相结合，构成了一个端到端的划痕检测模型。相较传统深度学习方法该模型可以在划痕先验知识的帮助下，可减轻对精标注样本的需求，有效减轻样本标注成本，在精标注的小样本集上即可完成训练。实验结果表明，所提出模型可有效检出光学元件表面的划痕，特别对弱划痕也具有好的检测效果，划痕检测精度高于相比较的其它传统分割方法和卷积神经网络。

(2)针对光学元件表面缺陷检测中样本精标注成本高，不满足质量快速评价的问题，提出了一种基于不确切监督的空间对抗卷积神经网络。该网络能够在图像级标注这种弱监督条件下，利用不确切监督中的多示例学习方法进行表面缺陷的分类和定位。该网络由特征提取模块和空间对抗模块两部分组成，分别用于表示多示例学习中的包和聚合函数。在特征提取模块中，把图像作为多示例学习里的包，用堆叠的卷积块输出的特征图中的元素表示包内的样本。在空间对抗模块中用多示例学习中的最大值聚合函数把特征图上具有最大概率值的元素作为竞争胜出对像引导网络进行训练，并输出缺陷分类和定位两个结果。在三个数据集上的实验表明，在只利用图像级标注数据的条件下，能够同时得到图像的分类结果和缺陷在图像上的具体位置，提高了网络的解释性。

(3)针对光学元件表面颗粒检测中标注样本少，无标注样本多的问题，提出一种利用无标注样本辅助神经网络训练的方法。该方法包括用于特征学习的自监督网络和颗粒物分类网络，以及基于特征复用的颗粒物检测三个部分。根据图像中颗粒与非颗粒物在旋转和翻转变换下类别不变的性质，提出一个基于旋转—翻转不变性的代理任务，用大量无标注的样本进行自监督网络的训练，把训练完成的底部卷积层作为特征提取器，迁移到颗粒物分类网络。随后，在迁移而来的特征提取器上加入逐点卷积层构建了对中心点进行分类的网络，使用小的标注样本集进行颗粒物分类网络的微调。最后，利用提取器和逐点卷积的特性，通过隐含层特征的复用使得颗粒物分类网络可以对图像上的所有像素完成分类，完成颗粒物的检测。实验表明，提出的基于自监督的颗粒物检测模型能够满足对颗粒物表面检测精度的需求。

(4)针对正样本条件下光学元件表面颗粒物检测问题，提出了一个基于图像修复和比较的两阶段颗粒物检测模型。该模型每个阶段分别由一个卷积神经网络构成，用于完成图像修复和比较两个功能。利用缺陷生成器产生缺陷样本和缺陷的真实分布作为监督信息，分别用于修复网络和比较网络的训练。在图像修复阶段，采用一个编码—解码结构的卷积神网络，用于把添加了人工伪造颗粒物的正样本图像进行修复，去除图像中人工伪造的颗粒，恢复为无颗粒的正样本。在图像比较阶段，构造了一个颗粒物比较网络用于进行待检测图像与其修复图像之间差异的比较，完成颗粒物的检测。通过实验表明，提出的基于正样本的方法能够在无监督的条件下检出大部分颗粒物，具有进一步研究的价值。

Optical elements are key components of various optical instruments, and their surface quality directly affects the performance of optical instruments. Defects on the surface of optical elements usually cause changes in the optical path, causing light scattering, changing the light transmission characteristics, and reducing the overall performance of the system, or damaging the optical elements themselves and the optical instruments, causing serious property losses or even accidents. It is important and necessary to detect defects on the surface of optical elements in the process of manufacturing, processing and use. The use of machine vision-based inspection methods can improve efficiency, reduce costs, and protect human health. It is significant for both industrial production and scientific research. This paper uses the deep learning method to focus on the key issues in the surface defect detection of optical elements. The main contributions of this dissertation are as follows:

(1)Aiming at the problem of accurate detection of weak scratches on the surface of optical elements, a scratch segmentation network incorporating prior knowledge is proposed. Through the analysis of the law of scratch imaging in dark field, the three characteristics of scratches in brightness, width and structure are obtained, and the local maximum index (LMI) and direction-sensitive convolution (DSC) are designed according to the three characteristics.  Two computable features, as a priori knowledge of scratches, LMI and DSC can significantly enhance the difference between scratches and background in dark-field images. By combining LMI and DSC with an encoding-decoding segmentation network, an end-to-end scratch detection model is formed. Compared with traditional deep learning methods, this model can reduce the need for fine-labeled samples and effectively reduce the cost of sample labeling with the help of prior knowledge of scratches. Training can be completed on a small sample set of fine-labeled. Experimental results show that the proposed model can effectively detect scratches on the surface of optical elements, especially for weak scratches. The detection accuracy of scratches is higher than that of other traditional segmentation methods and convolutional neural networks.

(2)Aiming at the high cost of precise labeling of samples in the surface defect detection of optical elements, which does not satisfy the problem of rapid quality evaluation, a spatial adversarial convolutional neural network based on imprecise supervision is proposed. The network can classify and locate surface defects by using the multi-instance learning  method in imprecise supervision under the weak supervision condition of image-level annotation. The network is composed of two parts: a feature extraction module and a spatial confrontation module, which respectively represent the packet and aggregation function in multi-instance learning. In the feature extraction module, the image is used as a package in multi-instance learning, and the elements in the feature map output by the stacked convolution block represent the samples in the package. In the spatial confrontation module, the maximum aggregation function in multi-instance learning is used to train the element with the largest probability value on the feature map as the competition winner, and output the two results of defect classification and positioning. Experiments on three datasets show that under the condition of only using image-level annotation data, the classification result of the image and the specific locations of the defects on the image can be obtained at the same time, which improves the interpretability of the network.

(3)Aiming at the problem that there are few labeled samples and many unlabeled samples in particle detection on the surface of optical elements, a method of using unlabeled samples to assist neural network training is proposed. The method includes three parts: a self-supervised network for feature learning, a particle classification network, and a particle detection part based on feature reuse. According to the category-invariant nature of the particles and non-particles in the image under rotation and flipping transformations, a surrogate task based on rotation-flip invariance is proposed. A large number of unlabeled samples are used to train the self-supervised network, and the bottom of the network as a feature extractor is transferred to the particle classification network. Subsequently, pointwise convolutional layers are added to the feature extractor to construct a network for classifying the center point, and a small labeled sample set is used to fine-tune the particle classification network. Finally, using the characteristics of the feature extractor and pointwise convolution, through the reuse of hidden layer features, the particle classification network can classify all pixels on the image and complete the detection of particles. Experiments show that the proposed self-supervised particle detection model can meet the requirements for the accuracy of particle surface detection.

(4)Aiming at the problem of particle detection on the surface of optical elements under the condition of positive samples, a two-stage particle detection model based on image restoration and comparison is proposed. Each stage of the model is composed of a convolutional neural network, which is used to complete the function of image restoration or image comparison. The defect generator is used to generate defect samples and the true distribution of defects as supervision information, which are used to train the restoration network and comparison network respectively. In the image restoration stage, a convolutional neural network with an encoding-decoding structure is used to restore the positive sample image with artificially forged particles, remove the artificially forged particles in the image, and restore to the positive sample without particles. In the image comparison stage, a particle detection network is constructed to compare the difference between the image to be detected and its restored image to complete the detection of particles. Experiments show that the proposed method based on positive samples can detect most particles under unsupervised conditions, which is of value for further research.