Knowledge Commons of Institute of Automation,CAS
Prior-knowledge and attention based meta-learning for few-shot learning | |
Qin, Yunxiao1; Zhang, Weiguo1; Zhao, Chenxu2; Wang, Zezheng3; Zhu, Xiangyu4; Shi, Jingping1; Qi, Guojun5; Lei, Zhen4,6 | |
发表期刊 | KNOWLEDGE-BASED SYSTEMS |
ISSN | 0950-7051 |
2021-02-15 | |
卷号 | 213页码:12 |
通讯作者 | Qin, Yunxiao(qyxqyx@mail.nwpu.edu.cn) |
摘要 | Recently, meta-learning has been shown to be a promising way to solve few-shot learning. In this paper, inspired by the human cognition process, which utilizes both prior-knowledge and visual attention when learning new knowledge, we present a novel paradigm of meta-learning approach that capitalizes on three developments to introduce attention mechanism and prior-knowledge to meta-learning. In our approach, prior-knowledge is responsible for helping the meta-learner express the input data in a high-level representation space, and the attention mechanism enables the meta-learner to focus on key data features in the representation space. Compared with the existing meta-learning approaches that pay little attention to prior-knowledge and visual attention, our approach alleviates the meta-learner's few-shot cognition burden. Furthermore, we discover a Task-Over-Fitting (TOF) problem,(1) which indicates that the meta-learner has poor generalization across different K-shot learning tasks. To model the TOF problem, we propose a novel Cross-Entropy across Tasks (CET) metric.(2) Extensive experiments demonstrate that our techniques improve the meta-learner to state-of-the-art performance on several few-shot learning benchmarks while also substantially alleviating the TOF problem. (C) 2020 Elsevier B.V. All rights reserved. |
关键词 | Meta-learning Few-shot learning Prior-knowledge Representation Attention mechanism |
DOI | 10.1016/j.knosys.2020.106609 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Key Research and Development Program of China[2020YFC2003901] ; National Natural Science Foundation of China[61573286] ; National Natural Science Foundation of China[61876178] ; National Natural Science Foundation of China[61976229] |
项目资助者 | National Key Research and Development Program of China ; National Natural Science Foundation of China |
WOS研究方向 | Computer Science |
WOS类目 | Computer Science, Artificial Intelligence |
WOS记录号 | WOS:000614644100011 |
出版者 | ELSEVIER |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/43351 |
专题 | 多模态人工智能系统全国重点实验室_生物识别与安全技术 |
通讯作者 | Qin, Yunxiao |
作者单位 | 1.Northwestern Polytech Univ, Xian 710129, Peoples R China 2.MiningLamp Technol, Beijing 100094, Peoples R China 3.Beijing Kwai Technol, Beijing 102600, Peoples R China 4.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit NLPR, Beijing 100000, Peoples R China 5.Huawei Cloud, Seattle, WA 90876 USA 6.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China |
推荐引用方式 GB/T 7714 | Qin, Yunxiao,Zhang, Weiguo,Zhao, Chenxu,et al. Prior-knowledge and attention based meta-learning for few-shot learning[J]. KNOWLEDGE-BASED SYSTEMS,2021,213:12. |
APA | Qin, Yunxiao.,Zhang, Weiguo.,Zhao, Chenxu.,Wang, Zezheng.,Zhu, Xiangyu.,...&Lei, Zhen.(2021).Prior-knowledge and attention based meta-learning for few-shot learning.KNOWLEDGE-BASED SYSTEMS,213,12. |
MLA | Qin, Yunxiao,et al."Prior-knowledge and attention based meta-learning for few-shot learning".KNOWLEDGE-BASED SYSTEMS 213(2021):12. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论