Prior-knowledge and attention based meta-learning for few-shot learning

doi:10.1016/j.knosys.2020.106609

CASIA OpenIR > 多模态人工智能系统全国重点实验室 > 生物识别与安全技术

	Prior-knowledge and attention based meta-learning for few-shot learning
	Qin, Yunxiao 1; Zhang, Weiguo 1; Zhao, Chenxu 2; Wang, Zezheng 3; Zhu, Xiangyu4 ; Shi, Jingping 1; Qi, Guojun 5; Lei, Zhen4,6
发表期刊	KNOWLEDGE-BASED SYSTEMS
ISSN	0950-7051
	2021-02-15
卷号	213 页码:12
通讯作者	Qin, Yunxiao(qyxqyx@mail.nwpu.edu.cn)
摘要	Recently, meta-learning has been shown to be a promising way to solve few-shot learning. In this paper, inspired by the human cognition process, which utilizes both prior-knowledge and visual attention when learning new knowledge, we present a novel paradigm of meta-learning approach that capitalizes on three developments to introduce attention mechanism and prior-knowledge to meta-learning. In our approach, prior-knowledge is responsible for helping the meta-learner express the input data in a high-level representation space, and the attention mechanism enables the meta-learner to focus on key data features in the representation space. Compared with the existing meta-learning approaches that pay little attention to prior-knowledge and visual attention, our approach alleviates the meta-learner's few-shot cognition burden. Furthermore, we discover a Task-Over-Fitting (TOF) problem,(1) which indicates that the meta-learner has poor generalization across different K-shot learning tasks. To model the TOF problem, we propose a novel Cross-Entropy across Tasks (CET) metric.(2) Extensive experiments demonstrate that our techniques improve the meta-learner to state-of-the-art performance on several few-shot learning benchmarks while also substantially alleviating the TOF problem. (C) 2020 Elsevier B.V. All rights reserved.
关键词	Meta-learning Few-shot learning Prior-knowledge Representation Attention mechanism
DOI	10.1016/j.knosys.2020.106609
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2020YFC2003901] ; National Natural Science Foundation of China[61573286] ; National Natural Science Foundation of China[61876178] ; National Natural Science Foundation of China[61976229]
项目资助者	National Key Research and Development Program of China ; National Natural Science Foundation of China
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000614644100011
出版者	ELSEVIER
引用统计	被引频次：18[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/43351
专题	多模态人工智能系统全国重点实验室_生物识别与安全技术
通讯作者	Qin, Yunxiao
作者单位	1.Northwestern Polytech Univ, Xian 710129, Peoples R China 2.MiningLamp Technol, Beijing 100094, Peoples R China 3.Beijing Kwai Technol, Beijing 102600, Peoples R China 4.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit NLPR, Beijing 100000, Peoples R China 5.Huawei Cloud, Seattle, WA 90876 USA 6.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Qin, Yunxiao,Zhang, Weiguo,Zhao, Chenxu,et al. Prior-knowledge and attention based meta-learning for few-shot learning[J]. KNOWLEDGE-BASED SYSTEMS,2021,213:12.
APA	Qin, Yunxiao.,Zhang, Weiguo.,Zhao, Chenxu.,Wang, Zezheng.,Zhu, Xiangyu.,...&Lei, Zhen.(2021).Prior-knowledge and attention based meta-learning for few-shot learning.KNOWLEDGE-BASED SYSTEMS,213,12.
MLA	Qin, Yunxiao,et al."Prior-knowledge and attention based meta-learning for few-shot learning".KNOWLEDGE-BASED SYSTEMS 213(2021):12.