Knowledge Commons of Institute of Automation,CAS
Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss | |
Li, Qianzhong1,2; Zhang, Yujia1; Sun, Shiying1; Zhao, Xiaoguang1; Li, Kang3; Tan, Min1 | |
发表期刊 | Neurocomputing |
ISSN | 0925-2312 |
2021-08-18 | |
卷号 | 449页码:117-135 |
摘要 | Zero-shot object detection (ZSD) aims to locate and recognize novel objects without additional training samples. Most existing methods usually map visual features to semantic space, resulting in a hubness problem, and learning an effective feature mapping between the two modalities remains a considerable challenge. In this work, we propose a novel end-to-end framework, Semantic-Visual Auto-Encoder (SVAE) network, to tackle the above issues. Distinct from previous works that utilize fully-connected layers to learn the feature mapping, we implement a 1-dimensional convolution with various shared filters to construct the auto-encoder, which maps semantic features to visual space to alleviate the hubness problem. Specifically, we design a novel loss function, Softplus Margin Focal Loss (SMFL), for object classification channel to align the projected semantic features in visual space and address the class imbalance problem. The SMFL improves the discrimination of projections on positive and negative categories and maintains the property of focal loss. Besides, to promote the localization performance for novel objects, we also provide semantic information for object localization channel and utilize a trainable matrix to align the semantic-visual mapping, considering noises in semantic representations. We conduct extensive experiments on four challenging benchmarks. The experimental results show the competitive performances compared with state-of-the-art approaches. Especially, we achieve 8.39%/6.58% mean average precision (mAP) improvements for ZSD/general-ZSD on Microsoft COCO benchmark. |
关键词 | Zero-shot object detection Softplus margin focal loss Semantic-visual alignment Auto-encoder architecture |
DOI | 10.1016/j.neucom.2021.03.073 |
关键词[WOS] | ATTRIBUTES |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Key Research and Development Project of China[2019YFB1310601] ; National Key R&D Program of China[2017YFC082020303] ; National Natural Science Foundation of China[61673378] |
项目资助者 | National Key Research and Development Project of China ; National Key R&D Program of China ; National Natural Science Foundation of China |
WOS研究方向 | Computer Science |
WOS类目 | Computer Science, Artificial Intelligence |
WOS记录号 | WOS:000652818400011 |
出版者 | ELSEVIER |
七大方向——子方向分类 | 多模态智能 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/45218 |
专题 | 复杂系统认知与决策实验室_先进机器人 |
通讯作者 | Li, Qianzhong |
作者单位 | 1.The State Key Laboratory of Management and Control for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2.School of Artificial Intelligences, University of Chinese Academy of Sciences, Beijing, China 3.Information Science Academy of China Electronics Technology Group Corporation, Beijing, China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Li, Qianzhong,Zhang, Yujia,Sun, Shiying,et al. Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss[J]. Neurocomputing,2021,449:117-135. |
APA | Li, Qianzhong,Zhang, Yujia,Sun, Shiying,Zhao, Xiaoguang,Li, Kang,&Tan, Min.(2021).Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss.Neurocomputing,449,117-135. |
MLA | Li, Qianzhong,et al."Rethinking semantic-visual alignment in zero-shot object detection via a softplus margin focal loss".Neurocomputing 449(2021):117-135. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Rethinking semantic-(8753KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论