Dynamic Warping Network for Semantic Video Segmentation

doi:10.1155/2021/6680509

	Dynamic Warping Network for Semantic Video Segmentation
	Li, Jiangyun 1,2; Zhao, Yikai 1; He, Xingjian3,4 ; Zhu, Xinxin3,4 ; Liu, Jing 4
发表期刊	COMPLEXITY
ISSN	1076-2787
	2021-02-08
卷号	2021 页码:10
通讯作者	Li, Jiangyun(leejy@ustb.edu.cn)
摘要	A major challenge for semantic video segmentation is how to exploit the spatiotemporal information and produce consistent results for a video sequence. Many previous works utilize the precomputed optical flow to warp the feature maps across adjacent frames. However, the imprecise optical flow and the warping operation without any learnable parameters may not achieve accurate feature warping and only bring a slight improvement. In this paper, we propose a novel framework named Dynamic Warping Network (DWNet) to adaptively warp the interframe features for improving the accuracy of warping-based models. Firstly, we design a flow refinement module (FRM) to optimize the precomputed optical flow. Then, we propose a flow-guided convolution (FG-Conv) to achieve the adaptive feature warping based on the refined optical flow. Furthermore, we introduce the temporal consistency loss including the feature consistency loss and prediction consistency loss to explicitly supervise the warped features instead of simple feature propagation and fusion, which guarantees the temporal consistency of video segmentation. Note that our DWNet adopts extra constraints to improve the temporal consistency in the training phase, while no additional calculation and postprocessing are required during inference. Extensive experiments show that our DWNet can achieve consistent improvement over various strong baselines and achieves state-of-the-art accuracy on the Cityscapes and CamVid benchmark datasets.
DOI	10.1155/2021/6680509
收录类别	SCI
语种	英语
资助项目	Fundamental Research Funds for the China Central Universities of USTB[FRF-DF-19-002] ; Scientific and Technological Innovation Foundation of Shunde Graduate School, USTB[BK20BE014]
项目资助者	Fundamental Research Funds for the China Central Universities of USTB ; Scientific and Technological Innovation Foundation of Shunde Graduate School, USTB
WOS研究方向	Mathematics ; Science & Technology - Other Topics
WOS类目	Mathematics, Interdisciplinary Applications ; Multidisciplinary Sciences
WOS记录号	WOS:000621847600002
出版者	WILEY-HINDAWI
引用统计	被引频次：3[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/43986
专题	紫东太初大模型研究中心_图像与视频分析
通讯作者	Li, Jiangyun
作者单位	1.Univ Sci & Technol Beijing, Sch Automat & Elect Engn, Beijing 100083, Peoples R China 2.Univ Sci & Technol Beijing, Shunde Grad Sch, Foshan 528300, Peoples R China 3.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100083, Peoples R China 4.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100083, Peoples R China
推荐引用方式 GB/T 7714	Li, Jiangyun,Zhao, Yikai,He, Xingjian,et al. Dynamic Warping Network for Semantic Video Segmentation[J]. COMPLEXITY,2021,2021:10.
APA	Li, Jiangyun,Zhao, Yikai,He, Xingjian,Zhu, Xinxin,&Liu, Jing.(2021).Dynamic Warping Network for Semantic Video Segmentation.COMPLEXITY,2021,10.
MLA	Li, Jiangyun,et al."Dynamic Warping Network for Semantic Video Segmentation".COMPLEXITY 2021(2021):10.