毕业生知识库

人类获取的信息绝大部分来自视觉，视觉是人类认识和改造世界的一个主要途径。虽然传统的二维成像技术已趋向成熟，但它们无法记录和重现出现实物理世界的三维信息。而三维成像技术可以突破以往二维技术的缺陷，更加真实地记录三维场景。

三维成像技术包括了三维场景的采集、处理以及显示。光场显示是未来的三维显示技术。光场显示的内容源于场景的采集与处理，采集与处理的场景信息需要光场显示技术才能重现。因此，光场显示的迅猛发展必然会促进采集、处理及相关技术的发展。然而国内外关于三维成像技术的研究与应用仍存在着诸多关键性问题，如：三维场景采集处理的质量与速度、三维显示的深度感与舒适度、显示质量等。针对这些问题，本文基于计算光场理论和平行系统对三维成像技术做优化，研究开发面向计算光场三维成像系统的场景采集、处理与显示，形成三维成像生成和展示优化平台。基于GPU开发出了高质量的多相机阵列采集、处理与显示实时系统，实现了由采集处理到显示再到优化的整个过程，并应用于光栅光场三维显示、移动端裸眼三维显示和VR显示等。本文主要贡献可以概括如下：

（1）计算三维采集与处理：改进了基于光场的深度图获取方法，在极线平面图像（EPI）上提取一致性线索，包括局部一致性和全局一致性。并通过光场重构提取重聚焦线索，结合虚拟合成孔径，提高重聚焦的易控性。最后，根据一致性线索和重聚焦线索，提出并实现了一种鲁棒性较高的自适应双线索深度估计方法；提出并实现了一种实用性较高的深度图优化方法，并通过基于深度图的视差扩张自由视点渲染算法，提高后续三维显示的立体感；通过最小化非凸正则能量函数，提出并实现了一种基于极线平面图像（EPI）的自由视点渲染方法，提高自由视点渲染的质量与速度；提出并实现了一种通过多视点冗余信息以及增加窗口的光场重构自由视点渲染方法，减少渲染时自由视点图像的模糊度。通过场景光场重构和非线性的聚焦搜索，提高搜索效率，并应用于六自由度VR显示。

（2）计算光场显示标定：通过有监督的学习，提出并实现了一种基于计算全局光场采集的优化方法对视点子像素进行分配标定，标定准确度显著提高；提出并实现了一种移动设备自标定方法，可标定任意未知光学参数的光栅面板（或微透镜阵列）以及未知子像素参数的移动设备，并应用于移动端的裸眼显示。

（3）计算光场显示优化：提出并实现了一种光线反跟踪和子像素权重融合算法，对子像素颜色进行校正，减少三维显示中的串扰现象，并实现了不同视点间的平滑融合；为消除显示时视点间的跳跃现象，提出并实现了一种子像素复用的超多视点渲染算法，并提供了显示参数的设计方案。通过解决有包围盒的整数最优化问题，进行复用渲染；为增大三维显示的观看角度，提出了一种分段分配标定的方法进行仿真，可结合人眼跟踪技术，增大观看角度。

（4）平行显示优化系统：提出了一种光场显示深度感知平行系统，综合考虑多种深度感知影响因素并建立显示评估模型，优化光场显示中视觉聚焦点位置、视频明暗对比度以及视频模糊度等，并进行相应地数据处理与修正，从而渲染出强深度感、高舒适度的三维显示；结合视频目标物体跟踪做深度感知优化，基于平行学习理论，提出一种训练数据增强算法，用少量真实图像产生多个虚拟训练图像，提高训练跟踪的准确度。为光场显示深度感知平行系统的实现提供了技术支持。

For human beings, vision is the most important source of information. Although the traditional 2D imaging technology has been developed maturely, it fails to record the 3D information of the physical world that humans are living in. However, the 3D imaging technology breaks the limits of traditional 2D technology and enhances the ability in scene reconstruction greatly.

 

The 3D imaging technology consists of the 3D scene acquisition, data processing and the display. Light field display is the future of the 3D display. The 3D data acquisition and processing are the major source of the content for 3D display, while the 3D data can’t be accurately presented without the 3D display. As a result, the rapid development of the light field display will inevitably promote the development of the 3D acquisition and processing technologies. However, there are still many key issues in the researches and applications of the 3D imaging. To solve those problems, this thesis dedicated to the research of the 3D imaging technology based on the computational light field theory and parallel system. This thesis developed a 3D imaging system and formed an optimization platform. The 3D imaging system, which included 3D acquisition, data processing, display and optimization, was realized. Using a GPU based processing algorithm, it could perform real-time 3D imaging with high accuracy. It was finally applied to the lenticular display, free-glass mobile display and the VR display. The main contributions are as below:

 

(1) Computational 3D acquisition and processing: The depth estimation algorithm based on the light field was improved. The disparity from the linear structure of the epi-polar plane image (EPI) was extracted both in the local viewpoints and the global viewpoints. Subsequently, all the rays in the light field were resampled for the digital refocus and an easy-controllable synthetic aperture resample method was provided. Finally, the scene depth map was recovered through a double-cues adaptive fusion (DAF) method; For the free viewpoint rendering based on the depth map, a practical depth map refinement algorithm was presented, and a disparity expansion method was also developed for the 3D stereoscopic enhancement; As to the free viewpoint rendering based on the EPI, a novel approach by minimizing a global spatially regularized non-convex energy function was implemented to improve both the rendering quality and speed; And for the free viewpoint rendering based on the light field, a blur reduction method through the light field reconstruction was proposed by using the redundant viewpoint information. Besides, through the rays reconstruction and nonlinear focus searching, the algorithm efficiency could be improved. Finally, it was applied to the 6-FoV VR display.

 

(2) Computational light field display calibration: Through a supervised learning optimization, a novel viewpoint subpixel calibration method based on the light field acquisition was implemented. The final calibration accuracy was improved significantly; For the calibration of the mobile devices, a turnkey solution to automatic calibration was proposed, which required no prior knowledge of microlens and mobile’s parameters. Finally, it was applied to the 3D mobile display.

 

(3) Computational light field display optimization: A novel subpixel color correction and crosstalk reduction method was proposed. A new algorithm of ray’s mergence and assignment was developed for the smooth fusion of different views; And a subpixel multiplexing method was implemented for the continuity optimization of the viewpoint through a box-constrained integer least squares algorithm. Besides, a method for display parameters design was also provided; Finally, a segmented-calibration method was proposed and it could increase the lenticular display's FoV by using the eye tracking technology.

 

(4) Parallel display optimization system: A novel 3D display parallel system for depth sense optimization was proposed and it empirically guided how the light field should be rendered; By virtual training data synthesis, a video processing parallel system for one-shot video segmentation was proposed to increase the segmentation accuracy, which provided the technical support for the realization of the parallel optimization system.