毕业生知识库

微光图像通常是在夜间等低照度条件下所获取的一类图像，具有对比度和亮度都很低的特点。智能手机、平板电脑、数字相机和航拍相机等设备的发展与广泛应用加剧了这类图像的产生。因其对比度和亮都很低，微光图像为后续处理带来很大难度。若是直接丢弃这类数据，人们可能会错失提取有用甚至重要信息的机会。微光图像增强的目标即是改善微光图像的质量，以方便人眼获得更好的视觉效果，或者更利于提升后续进行的检测、跟踪或识别等任务的效果。目前，无论是民用还是军用场合，存在着大量的微光图像，而传统图像增加方法在这类图像上难以取得令人满意的效果，因此展开针对微光图像增强的理论与方法研究具有十分重大的应用价值。

 

本文结合机器学习与模式识别理论对微光图像增强问题进行了建模，通过分析微光图像增强问题的特点，提出了三种微光图像增强方法。本文的研究内容主要包括：

 

1)提出一种基于样例学习的方法来解决微光图像增强问题。该方法将微光图像增强问题转化为一系列图像块的增强问题，运用耦合字典对问题进行建模。具体地，通过模型训练，该方法可以学得用于表示增强前后图像块的一对字典以及图像块在字典对下表示的线性映射矩阵。 对新到来的微光图像块，该方法利用在微光图像块字典上学得的表示来估计增强图像块。为了提升方法的鲁棒性，方法采用了分治的策略，引入了聚类的方法来将数据空间划分成K个簇，然后在单个簇内按上述做法来建模簇内的微光图像块增强问题。同时，为了加快运行速度，本文也提出一个简约快速模型。通过在大量微光图像的测试结果表明，该方法能十分有效的增强该类图像。相对以往方法，该方法在图像对比度和整体亮度自然度上能取得更好的平衡。另外，快速模型在损失部分细节增强的代价下，能极大地提高方法的运行速度。

 

2)提出基于模式回归的方法来解决微光图像增强问题。本文首先运用分片线性回归去直接估计了微光图像块与增强结果图像块之间的函数关系。为了放松局部线性约束，本文进一步提出一个考虑增强损失的随机森林回归方法。考虑到分片线性运用聚类进行分治时聚类数设置对结果的影响，且前两个方法均只能处理固定尺寸的图像块，于是本文提出运用多尺度混合线性回归的方法来解决微光图像问题，该方法将单个簇回归推广到利用多个簇回归，同时考虑了重构损失以及图像的多尺度信息。 在一个公开实验数据集和大量手工收集的微光图像数据集的实验表明，和经典方法相比，基于局线回归和基于随机森林的增强方法在峰值信噪比以及结构相似度指标上分别取得了较优的排名，而且基于随机森林的方法在视觉增强质量以及计算效率上要更优于局部线性方法。实验也证明了利用多尺度和混合线性的结合能进一步提升增强质量。

 

3)为了能满足实时应用要求，本文在假设拥有图形处理器的前提下，利用神经网络出色拟合任意复杂函数的能力，设计了一个端到端的深度卷积神经网络来解决微光图像增强问题，并探讨了以往方法与深度卷积神经网络的联系。通过实验表明，基于端到端的深度卷积神经网络在数值指标上优势明显，在大量测试图上视觉效果优异，表现稳定，而且运行速度极快，已能满足实时处理微光图像的应用需求。

Low Light Level Images (LLLIs), with the characteristics of extremely low brightness and low contrast, are usually captured in the environment of low light, for example during the night. The appearance and development of the devices including smart phone, table PC, digital cameras and aerial cameras, produces more and more LLLIs. It is difficult to directly make use of the LLLIs due to their poor brightness and low contrast. If we make no use of LLLIs, we may lose the opportunity to obtain useful or even important information. The purpose of LLLIs enhancement is to improve the quality of LLLIs for both human visual effects and automatic analysis application, such as  object detection, tracking or recognition and so on. Up to now, there exists a large quantity of LLLIs in daily and military life, so it is of both theoretical and applicative importance to conduct the technique researches on the LLLI enhancement.

 

We model the problem of LLLIs enhancement with the theories of machine learning and pattern recognition, and put forward three methods with the analysis on the characteristics of LLLIs enhancement problem:

 

1) We propose a LLLI enhancement method in the view of example based learning. The method decomposes the problem of a LLLI enhancement into settling the image patches of this LLLI, and model the problem with a coupled dictionary learning method. During the training stages, a pair of dictionaries and a linear mapping function are learned simultaneously. The dictionary pair aims to describe the raw LLLIs and their enhanced versions, and the linear mapping function models the correspondence between the representations of the dictionary pair. In the enhancement process, the resulting image is generated through dictionary mapping from patches of the input LLLI. We adopt the divide-and-conquer strategy with clustering to improve the robustness of coupled dictionary learning. To accelerate the enhancement process, we also propose an improved algorithm for fast implementation. The experiments on a large quantity of LLLIs demonstrate that the effectiveness of our methods. Compared with classical methods, our methods achieve a good balance between overall brightness and contrast. The fast version method runs much quicker than the original one at the cost of the loss of details recovery.

 

2) We propose three methods based on the pattern regression to settle the LLLI enhancement problem. We firstly formulate the LLLI enhancement as a locally linear regression problem: the regressor maps patches of input image to enhanced patches,

and the regression function is estimated by learning from sample images. To relax the locally linear restriction, we further propose a method on the basis of random forest with the considering on the loss of reconstruction. We also propose a multi-scale mixture linear regression method to reduce the effect of the cluster number on the enhancement results of the locally linear regression method, and make use of the multi-scale information and loss of reconstruction to improve the enhancement effect. Experimental results on an open data-set and practical LLLIs demonstrate the effectiveness of our methods. Both of the locally linear regression the random forest methods achieve a good ranking at the index of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). The random forest method performs superiorly in both enhancement quality and computation efficiency. The experiments also prove that the combination of multi-scale information and mixture linear regression can improve the enhancement results.

 

3) To fulfill the requirement of practical use with the Graphical Processing Unit (GPU), we propose a end-to-end deep convolutional neural network to enhance the LLLI, and the reason is the great power of neural network to fit any complex function. We discuss the connections between the deep convolutional neural network and the classical methods.

Experimental results demonstrate the method achieve the best place at the PSNR and SSIM, and the enhancement visual quality of the method is quite pleasing. Furthermore, the method performs robustly on the test image set, and runs extremely fast to fulfill the real time application.