Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision

doi:10.1109/TPAMI.2019.2911936

	Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision
	Liu, Si 1; Ren, Guanghui 3; Sun, Yao 3; Wang, Jinqiao2 ; Wang, Changhu 4; Li, Bo 1; Yan, Shuicheng 5
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
	2022-02-01
卷号	44 期号:2 页码:610-621
通讯作者	Wang, Jinqiao(jqwang@nlpr.ia.ac.cn)
摘要	In this paper, we target at the Fine-grAined human-Centric Tracklet Segmentation (FACTS) problem, where 12 human parts, e.g., face, pants, left-leg, are segmented. To reduce the heavy and tedious labeling efforts, FACTS requires only one labeled frame per video during training. The small size of human parts and the labeling scarcity makes FACTS very challenging. Considering adjacent frames of videos are continuous and human usually do not change clothes in a short time, we explicitly consider the pixel-level and frame-level context in the proposed Temporal Context segmentation Network (TCNet). On the one hand, optical flow is on-line calculated to propagate the pixel-level segmentation results to neighboring frames. On the other hand, frame-level classification likelihood vectors are also propagated to nearby frames. By fully exploiting the pixel-level and frame-level context, TCNet indirectly uses the large amount of unlabeled frames during training and produces smooth segmentation results during inference. Experimental results on four video datasets show the superiority of TCNet over the state-of-the-arts. The newly annotated datasets can be downloaded via http://liusi-group.com/projects/FACTS for the further studies.
关键词	Labeling Object segmentation Image segmentation Task analysis Semantics Training Face Video object segmentation human-centric fine-grained optical flow estimation
DOI	10.1109/TPAMI.2019.2911936
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2016YFC0801003] ; Natural Science Foundation of China[U1536203] ; Natural Science Foundation of China[61572493] ; Natural Science Foundation of China[61876177] ; Natural Science Foundation of China[61772527] ; Natural Science Foundation of China[61806200]
项目资助者	National Key R&D Program of China ; Natural Science Foundation of China
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000740006100006
出版者	IEEE COMPUTER SOC
七大方向——子方向分类	图像视频处理与分析
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/47191
专题	紫东太初大模型研究中心_图像与视频分析
通讯作者	Wang, Jinqiao
作者单位	1.Beihang Univ, Sch Comp Sci & Engn, Beijing 100191, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Informat Engn, Beijing 100190, Peoples R China 4.ByteDance AI Lab, Beijing, Peoples R China 5.Qihoo 360 AI Inst, Beijing, Peoples R China
通讯作者单位	中国科学院自动化研究所
推荐引用方式 GB/T 7714	Liu, Si,Ren, Guanghui,Sun, Yao,et al. Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2022,44(2):610-621.
APA	Liu, Si.,Ren, Guanghui.,Sun, Yao.,Wang, Jinqiao.,Wang, Changhu.,...&Yan, Shuicheng.(2022).Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,44(2),610-621.
MLA	Liu, Si,et al."Fine-Grained Human-Centric Tracklet Segmentation with Single Frame Supervision".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 44.2(2022):610-621.