Knowledge Commons of Institute of Automation,CAS
Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation | |
Fan, Junsong1,2![]() ![]() | |
发表期刊 | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
![]() |
ISSN | 0162-8828 |
2023-05-01 | |
卷号 | 45期号:5页码:6006-6020 |
通讯作者 | Zhang, Zhaoxiang(zhaoxiang.zhang@ia.ac.cn) |
摘要 | Weakly supervised semantic segmentation (WSSS) trains segmentation models by only weak labels, aiming to save the burden of expensive pixel-level annotations. This paper tackles the WSSS problem of utilizing image-level labels as the weak supervision. Previous approaches address this problem by focusing on generating better pseudo-masks from weak labels to train the segmentation model. However, they generally only consider every single image and overlook the potential cross-image contexts. We emphasize that the cross-image contexts among a group of images can provide complementary information for each other to obtain better pseudo-masks. To effectively employ cross-image contexts, we develop an end-to-end cross-image context module containing a memory bank mechanism and a transformer-based cross-image attention module. The former extracts cross-image contexts online from the feature encodings of input images and stores them as the memory. The latter mines useful information from the memorized contexts to provide the original queries with additional information for better pseudo-mask generation. We conduct detailed experiments on the Pascal VOC 2012 and the COCO dataset to demonstrate the advantage of utilizing cross-image contexts. Besides, state-of-the-art performance is also achieved. Codes are available at https://github.com/js-fan/MCIC.git. |
关键词 | Image segmentation Semantics Training Heating systems Context modeling Task analysis Computational modeling Weakly-supervised learning semantic segmentation image relationship cross-image context |
DOI | 10.1109/TPAMI.2022.3203402 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | Major Project for New Generation of AI[2018AAA0100400] ; National Natural Science Foundation of China[61836014] ; National Natural Science Foundation of China[U21B2042] ; National Natural Science Foundation of China[62072457] ; National Natural Science Foundation of China[62006231] ; InnoHK program |
项目资助者 | Major Project for New Generation of AI ; National Natural Science Foundation of China ; InnoHK program |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000964792800042 |
出版者 | IEEE COMPUTER SOC |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/53249 |
专题 | 多模态人工智能系统全国重点实验室 |
通讯作者 | Zhang, Zhaoxiang |
作者单位 | 1.Chinese Acad Sci HKISI CAS, Hong Kong Inst Sci & Innovat, Ctr Artificial Intelligence & Robot, Hong Kong 999077, Peoples R China 2.Chinese Acad Sci CASIA, Inst Automat, Ctr Res Intelligent Percept & Comp CRIPAC, Natl Lab Pattern Recognit NLPR, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci UCAS, Beijing 100190, Peoples R China |
第一作者单位 | 模式识别国家重点实验室 |
推荐引用方式 GB/T 7714 | Fan, Junsong,Zhang, Zhaoxiang. Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(5):6006-6020. |
APA | Fan, Junsong,&Zhang, Zhaoxiang.(2023).Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(5),6006-6020. |
MLA | Fan, Junsong,et al."Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.5(2023):6006-6020. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论