毕业生知识库

虚拟数字人生成有着极大的应用价值，有望革新电影、游戏和虚拟现实等多

个产业。在实际应用中，控制生成的虚拟数字人的姿势是一项自然而然的需求。

这种需求推动了两个重要的研究子领域的发展，分别是二维空间上的姿势引导

人体图像生成和三维空间上的姿势可控虚拟数字人生成。本文采用生成对抗网

络（GAN）方法，围绕这两个研究子领域进行研究。

二维空间上，姿势引导的人体图像生成是图像生成的重要子领域。在低分

辨率场景下（256 × 256），现有的姿势引导的人体图像生成方法已经取得重要进

展。目前需要把更多注意力放在高分辨率场景上，比如分辨率 512 × 512。然而，

该领域现有数据集和现有方法都有局限性。对此，本文提出了一个全新数据集

PersonHD，一个大规模的高分辨率的具有复杂人体姿势的人体图像数据集，并

建立了可用于评估现有方法的实验基准。针对高分辨率场景，本文提出了一个

全新的高分辨率人体图像生成框架，包括语义信息引导的人体部件级数据增强、

条件上采样模块、单样本的个性化驱动技巧。实验结果表明，本框架可以生成不

错的高分辨率人体图像。

三维空间上，姿势可控的 3D 数字人生成领域是 3D 生成领域的重要子领域。

生成穿着复杂服装和具有复杂姿态的 3D 虚拟人目前仍是具有极大挑战性的难

题。本文提出了一种方法 PC-GET，用于生成姿势可控的 3D 纹理网格形式的虚

拟数字人。本方法的关键点在于如何使用给定人体姿态形状信息来引导虚拟数

字人的纹理网格生成过程。本文探索了两种方式去使用参数化人体模型的姿势

形状信息，分别是基于人体模型进行神经纹理光栅化得到正交三平面特征和提

取人体模型对应点云特征。第一种方式，本文利用粗糙人体模型将生成的神经纹

理光栅化到三个正交特征平面上，得到光栅化正交三平面特征，从而显示引导了

数字人生成。第二种方式，本文使用粗糙的参数化人体模型对应的点云作为输

入，经过点云处理网络提取隐式特征来引导数字人生成。最后，本文对方法进行

实验评测，实验结果显示本文的方法 PC-GET 能够生成姿态可控的数字人 3D 纹

理网格表示。方法 PC-GET 生成的 3D 数字人纹理网格表示可以直接被 3D 渲染

引擎使用，迅速部署于新的光照场景，因而具有极大的实际应用价值。

本文的研究结果为姿势可控的 2D/3D 数字人人体生成做出了贡献，该成果

可用于虚拟现实、游戏和电影等各个领域。

The development of digital humans has the potential to revolutionize many industries, including film, gaming, and virtual reality. There exists a practical need of controlling the pose of generated virtual digital humans. This need has given rise to two important subfields of research, person image generation in 2D space and pose controllable digital human generation in 3D space.

Pose-guided person image generation is an important subarea of image generation. Recent works have achieved great success in pose-guided person image generation tasks with low-definition scenes. Nowadays, more attention needs to be paid to high-definition human transfer. However, existing datasets and methods of human pose transfer are limited in the high definition. In this paper, we introduce PersonHD, a large-scale high-definition person image dataset with comprehensive supplementary information (high-quality skeleton keypoints and semantic human parsing maps). Following the common setting of existing datasets, we establish comprehensive benchmarks to provide a new platform for evaluating recent state-of-the-art methods. Moreover, we propose a unified framework for high-resolution person image generation with the semantic enhanced part-wise augmentation and the conditional up-sampling module. Theexperimental results show that our pipeline produces decent results.

In the 3D space, pose-controllable digital human generation is an important subfield of the 3D human generation. Generating 3D virtual humans wearing complex clothing and with complex poses still remains a highly challenging problem. We propose the innovative Pose Controllable human textured mesh generator (PC-GET) that allows to synthesize 3D human textured mesh with the given pose. The keypoint is how to use the prior information of human pose to guide the generation process. We use a parametric human body template to represent a given human pose and shape and explore two independent ways to generate a digital human with the given pose, the rasterization tri-plane feature based on the template and the coarse point cloud feature representation. First, we obtain the orthogonal-viewed tri-plane feature through rasterization based on the template to explicitly guide the generation. Second, we use the coarse point cloud of the template as the input and extract the feature to implicitly guide the generation. Experimental results show that our model PC-GET is capable of generating pose-controllable 3D human textured mesh. The human textured mesh format canbe directly used by 3D rendering engines and is deployable to new lighting scenarios,thus suitable for downstream application.

The results of this paper contribute to pose-controllable 2D/3D digital human body generation, which can be used in various fields such as virtual reality, games, and film