毕业生知识库

目标分割是图像处理和计算机视觉领域中的热点研究问题。目标分割不仅在图像编辑和图像理解等任务中发挥着关键作用，同时也是构建诸多计算机视觉应用系统的重要基础。然而，对于精细结构目标，现有方法大多难以获得令人满意的分割结果。精细结构目标分割问题还远未得到有效解决，目前仍是一个具有挑战性的研究课题。因此，开展精细结构目标分割的研究具有十分重要的理论意义和应用价值。

 

精细结构目标分割主要存在如下两个难点：1）像素的标记信息难以从目标主体传播至局部细节，导致细节部分与目标主体相互脱离；2）精细结构目标不满足边界平滑性假设，导致现有方法难以获得高精度的分割结果。另外，精细结构目标数据集、分割模型和评估方法的缺乏，也阻碍了相关方法的研制。围绕这些问题，本文将开展精细结构目标分割研究，并提出多种有效的分割方法。论文的主要贡献包含以下几个方面：

 

1. 提出了一种基于局部与非局部邻域传播的分割方法。其核心思想是将标记传播过程定义在一种新建立的图模型上，即每个节点不仅与其相邻的节点建立局部连接，还与跨越更大范围的节点建立非局部连接。这种图模型能使目标的局部结构通过局部连接得到准确保持，同时也有利于标记信息通过非局部连接有效地传播到目标的细长部分。此外，该方法根据图像的表观模型定义区域损失并显式地加入到标记传播模型中，以便于更准确地分割标记信息难以传播的细长部分。大量实验从多个角度证实，提出的图模型适合于处理精细结构目标分割问题，并且该方法也取得了优于现有方法的分割结果。另外，针对于目前该领域内数据匮乏的问题，本文建立了一个包含100幅图像的精细结构目标图像数据集，以像素级精度手工标注了其中的精细结构目标的真实分割，为该领域内的方法评估和对比提供了便利，有利于该领域的快速发展。

 

2. 提出了一种基于数据引导随机游走的分割方法，并将其应用到多种类型的分割问题中。该方法从给定数据中学习具有目标/背景区分性的信息，并将据此定义的数据引导正则项作为对随机游走概率偏差的惩罚。 通过对随机游走粒子（random walkers）的引导，使其更容易在目标的细长部分中转移，以便于更准确地分割精细结构目标。在此基础上，针对于目标/背景二类分割和多类目标分割两种类型的问题，分别给出了有约束和无约束两种情况下能量函数最小化问题的闭合形式全局最优解。根据给定数据形式的不同，该方法可以应用到交互式目标/背景二类分割，相似图像间的传递分割，交互式多类目标分割，以及与深层卷积神经网络相结合的语义分割问题中。此外，数据引导作用的加入，提高了该方法对交互数量和位置的鲁棒性，使其善于处理结构复杂和不连通的目标，并且降低了对人工交互的依赖性。实验结果证实了该方法对精细结构目标分割的有效性。

 

3. 提出了一种基于边缘引导图切割的低交互量分割方法。首先，该方法针对精细结构目标的特点提出了一种简化的矩形框交互方式，并结合提出的一种基于像素密度估计的迭代式算法，自动生成能够取代笔划交互的目标和背景种子点。这种交互方式不仅降低了人工交互的负担，而且由于自动生成的种子点数目远比笔划交互提供的多，有利于更准确地计算图像的颜色分布。其次，为了解决图切割方法的“收缩偏差”（shrinking bias）问题，提出了一种边缘引导图切割方法。该方法利用图像的边缘信息，对马尔可夫随机场中二元势函数的权重进行自适应的调整，以此来避免因精细结构目标的边界过长致使切割代价过大而造成的“收缩偏差”问题。实验表明所提交互方式适合结构细长、延伸范围广的目标分割问题，并且具有较低的交互代价。

Object segmentation is undoubtedly one of the most fundamental tasks in image processing and computer vision. It not only plays a pivotal role in image editing and understanding, but also serves as an important foundation to construct various application systems of computer vision. However, when tackling images containing fine-structured (FS) objects, most existing methods fail to produce satisfactory results. Therefore, FS object segmentation is far from being well solved, and it is still a highly challenging problem. For these reasons, FS object segmentation is of great importance in both theoretical research and practical application.

 

Currently, the major challenges of FS object segmentation typically lie in the following two aspects. 1) Due to the complex object structure, label information is hard to propagate from the main body to the local fine parts, making the fine parts disconnected from the main body. 2) Most FS objects do not satisfy the assumption of object boundary smoothness, thus the segmentation quality of most existing methods is largely degraded. Moreover, the lack of data sets, segmentation models and evaluation methods largely increase the difficulty to develop new methods for FS object segmentation. To address these issues, this dissertation focuses on FS object segmentation and presents several effective methods. The main contributions are summarised as follows.

 

1. A novel method based on local and nonlocal neighborhood propagation is proposed. The core idea is to formulate label propagation on a specially constructed graph model, with each node connected to not only nearby local neighbors but also nonlocal neighbors that are faraway from it. In this model, local structures of the object can be preserved by local connections, and label information is encouraged to propagate along the fine structures via nonlocal connections. Moreover, region cost is explicitly incorporated into the label propagation framework. The region cost is defined based on image appearance models, and facilitates the segmentation of the fine structures where label information is hard to propagate correctly. Extensive experiments demonstrate the applicability of the graph model for FS object segmentation and the superior performance of our method. In addition, this dissertation creates a novel data set consisting of 100 images, each containing one or a few FS objects. Accurately hand-labeled ground truth is provided for each FS object. This data set largely enriches the method evaluation and comparison in this field, and is beneficial to further researches.

 

2. A data-guided random walks method is proposed and applied to several kinds of segmentation problems involving FS objects. This method learns the separability between objects and backgrounds based on certain types of given data. A data-guided regularization term is defined hereby, to penalize the deviations of random walk probabilities. Under such guidance, the random walkers are enabled to transfer more easily in the fine structures, yielding more accurate segmentation. For both binary and multi-label segmentation tasks, the globally optimal closed-form solutions are provided for the energy minimization problem both with and without constraints. According to the types of the given data, our method can be applied to interactive binary segmentation, transferring segmentation between similar images, interactive multi-label segmentation, as well as semantic segmentation when combined with deep convolutional neural network. Moreover, the integration of the data guidance makes the method more robust to user interactions and more powerful to tackle complex and disconnected objects. Comprehensive experiments verify the effectiveness and efficiency of our method.

 

3. An edge-guided graph cuts method with a simplified interaction paradigm is proposed. In this method, loosely dragged rectangles are used as interactions in place of the commonly used scribbles. Based on the interactions, an iterative algorithm is developed to generate plentiful object and background seeds. This paradigm not only reduces the interaction burdens, but also facilitates learning more accurate color distributions due to the larger number of seeds in comparison to scribbles. Furthermore, an edge-guided graph cuts method is proposed to mitigate the shrinking bias in graph cuts based methods. The core idea is to adaptively adjust the weightings of the pairwise potentials in a Markov random field according to image edges. In this way, the cost of cutting the long boundaries of FS objects is largely reduced, which is beneficial to the segmentation of FS objects. Experimental results show that the proposed interaction paradigm is suitable for tackling FS objects that span over the entire images with relatively lower interaction burdens.