Learning 3D Human Shape and Pose from Dense Body Parts | |
Zhang, Hongwen1,2; Cao, Jie1,2; Lu, Guo3; Ouyang, Wanli4; Sun, Zhenan1,2 | |
发表期刊 | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE |
2022 | |
卷号 | 44期号:5页码:2610-2627 |
摘要 | Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods. The commonly occurred misalignment comes from the facts that the mapping from images to the model space is highly non-linear and the rotation-based pose representation of body models is prone to result in the drift of joint positions. In this work, we investigate learning 3D human shape and pose from dense correspondences of body parts and propose a Decompose-and-aggregate Network (DaNet) to address these issues. DaNet adopts the dense correspondence maps, which densely build a bridge between 2D pixels and 3D vertices, as intermediate representations to facilitate the learning of 2D-to-3D mapping. The prediction modules of DaNet are decomposed into one global stream and multiple local streams to enable global and fine-grained perceptions for the shape and pose predictions, respectively. Messages from local streams are further aggregated to enhance the robust prediction of the rotation-based poses, where a position-aided rotation feature refinement strategy is proposed to exploit spatial relationships between body joints. Moreover, a Part-based Dropout (PartDrop) strategy is introduced to drop out dense information from intermediate representations during training, encouraging the network to focus on more complementary body parts as well as neighboring position features. The efficacy of the proposed method is validated on both indoor and real-world datasets including Human3.6M, UP3D, COCO, and 3DPW, showing that our method could significantly improve the reconstruction performance in comparison with previous state-of-the-art methods. Our code is publicly available at https://hongwenzhang.github.io/dense2mesh. |
关键词 | 3D human shape and pose estimation decompose-and-aggregate network position-aided rotation feature refinement part-based dropout |
收录类别 | SCI |
语种 | 英语 |
WOS记录号 | WOS:000792921400029 |
七大方向——子方向分类 | 模式识别基础 |
国重实验室规划方向分类 | 多尺度信息处理 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/44737 |
专题 | 模式识别实验室 |
通讯作者 | Sun, Zhenan |
作者单位 | 1.中国科学院自动化研究所 2.中国科学院大学 3.上海交通大学 4.悉尼大学 |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhang, Hongwen,Cao, Jie,Lu, Guo,et al. Learning 3D Human Shape and Pose from Dense Body Parts[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2022,44(5):2610-2627. |
APA | Zhang, Hongwen,Cao, Jie,Lu, Guo,Ouyang, Wanli,&Sun, Zhenan.(2022).Learning 3D Human Shape and Pose from Dense Body Parts.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,44(5),2610-2627. |
MLA | Zhang, Hongwen,et al."Learning 3D Human Shape and Pose from Dense Body Parts".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 44.5(2022):2610-2627. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
TPAMI20-Learning 3D (4759KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论