Deep Self-Supervised Representation Learning for Free-Hand Sketch

doi:10.1109/TCSVT.2020.3003048

CASIA OpenIR > 模式识别实验室

	Deep Self-Supervised Representation Learning for Free-Hand Sketch
	Xu, Peng 1; Song, Zeyu 2; Yin, Qiyue3 ; Song, Yi-Zhe 4; Wang, Liang3
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
	2021-04-01
卷号	31 期号:4 页码:1503-1513
通讯作者	Xu, Peng(peng.xu@ntu.edu.sg)
摘要	In this paper, we tackle for the first time, the problem of self-supervised representation learning for free-hand sketches. This importantly addresses a common problem faced by the sketch community - that annotated supervisory data are difficult to obtain. This problem is very challenging in which sketches are highly abstract and subject to different drawing styles, making existing solutions tailored for photos unsuitable. Key for the success of our self-supervised learning paradigm lies with our sketch-specific designs: (i) we propose a set of pretext tasks specifically designed for sketches that mimic different drawing styles, and (ii) we further exploit the use of the textual convolution network (TCN) together with the convolutional neural network (CNN) in a dual-branch architecture for sketch feature learning, as means to accommodate the sequential stroke nature of sketches. We demonstrate the superiority of our sketch-specific designs through two sketch-related applications (retrieval and recognition) on a million-scale sketch dataset, and show that the proposed approach outperforms the state-of-the-art unsupervised representation learning methods, and significantly narrows the performance gap between with supervised representation learning. (1) (1) PyTorch code of this work is available at https://github.com/zzz1515151/self-supervised_learning_sketch.
关键词	Feature extraction Task analysis Strain Computer architecture Deep learning Deformable models Convolution Self-supervised representation learning deep learning sketch pretext task textual convolution network convolutional neural network
DOI	10.1109/TCSVT.2020.3003048
收录类别	SCI
语种	英语
资助项目	BUPT Excellent Ph.D. ; Student Foundation[CX2017307] ; BUPT-SICE Excellent Graduate Student Innovation Foundation
项目资助者	BUPT Excellent Ph.D. ; Student Foundation ; BUPT-SICE Excellent Graduate Student Innovation Foundation
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:000637537200021
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：24[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/44249
专题	模式识别实验室复杂系统认知与决策实验室_智能系统与工程
通讯作者	Xu, Peng
作者单位	1.Nanyang Technol Univ, Sch Comp Sci & Engn, Singapore 639798, Singapore 2.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing 100876, Peoples R China 3.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 4.Univ Surrey, Ctr Vis Speech & Signal Proc, Guildford GU2 7XH, Surrey, England
推荐引用方式 GB/T 7714	Xu, Peng,Song, Zeyu,Yin, Qiyue,et al. Deep Self-Supervised Representation Learning for Free-Hand Sketch[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2021,31(4):1503-1513.
APA	Xu, Peng,Song, Zeyu,Yin, Qiyue,Song, Yi-Zhe,&Wang, Liang.(2021).Deep Self-Supervised Representation Learning for Free-Hand Sketch.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,31(4),1503-1513.
MLA	Xu, Peng,et al."Deep Self-Supervised Representation Learning for Free-Hand Sketch".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 31.4(2021):1503-1513.