CASIA OpenIR  > 多模态人工智能系统全国重点实验室
Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation
Fan, Junsong1,2; Zhang, Zhaoxiang1,3
Source PublicationIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN0162-8828
2023-05-01
Volume45Issue:5Pages:6006-6020
Corresponding AuthorZhang, Zhaoxiang(zhaoxiang.zhang@ia.ac.cn)
AbstractWeakly supervised semantic segmentation (WSSS) trains segmentation models by only weak labels, aiming to save the burden of expensive pixel-level annotations. This paper tackles the WSSS problem of utilizing image-level labels as the weak supervision. Previous approaches address this problem by focusing on generating better pseudo-masks from weak labels to train the segmentation model. However, they generally only consider every single image and overlook the potential cross-image contexts. We emphasize that the cross-image contexts among a group of images can provide complementary information for each other to obtain better pseudo-masks. To effectively employ cross-image contexts, we develop an end-to-end cross-image context module containing a memory bank mechanism and a transformer-based cross-image attention module. The former extracts cross-image contexts online from the feature encodings of input images and stores them as the memory. The latter mines useful information from the memorized contexts to provide the original queries with additional information for better pseudo-mask generation. We conduct detailed experiments on the Pascal VOC 2012 and the COCO dataset to demonstrate the advantage of utilizing cross-image contexts. Besides, state-of-the-art performance is also achieved. Codes are available at https://github.com/js-fan/MCIC.git.
KeywordImage segmentation Semantics Training Heating systems Context modeling Task analysis Computational modeling Weakly-supervised learning semantic segmentation image relationship cross-image context
DOI10.1109/TPAMI.2022.3203402
Indexed BySCI
Language英语
Funding ProjectMajor Project for New Generation of AI[2018AAA0100400] ; National Natural Science Foundation of China[61836014] ; National Natural Science Foundation of China[U21B2042] ; National Natural Science Foundation of China[62072457] ; National Natural Science Foundation of China[62006231] ; InnoHK program
Funding OrganizationMajor Project for New Generation of AI ; National Natural Science Foundation of China ; InnoHK program
WOS Research AreaComputer Science ; Engineering
WOS SubjectComputer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS IDWOS:000964792800042
PublisherIEEE COMPUTER SOC
Citation statistics
Cited Times:3[WOS]   [WOS Record]     [Related Records in WOS]
Document Type期刊论文
Identifierhttp://ir.ia.ac.cn/handle/173211/53249
Collection多模态人工智能系统全国重点实验室
Corresponding AuthorZhang, Zhaoxiang
Affiliation1.Chinese Acad Sci HKISI CAS, Hong Kong Inst Sci & Innovat, Ctr Artificial Intelligence & Robot, Hong Kong 999077, Peoples R China
2.Chinese Acad Sci CASIA, Inst Automat, Ctr Res Intelligent Percept & Comp CRIPAC, Natl Lab Pattern Recognit NLPR, Beijing 100190, Peoples R China
3.Univ Chinese Acad Sci UCAS, Beijing 100190, Peoples R China
First Author AffilicationChinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Fan, Junsong,Zhang, Zhaoxiang. Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(5):6006-6020.
APA Fan, Junsong,&Zhang, Zhaoxiang.(2023).Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(5),6006-6020.
MLA Fan, Junsong,et al."Memory-Based Cross-Image Contexts for Weakly Supervised Semantic Segmentation".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.5(2023):6006-6020.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Fan, Junsong]'s Articles
[Zhang, Zhaoxiang]'s Articles
Baidu academic
Similar articles in Baidu academic
[Fan, Junsong]'s Articles
[Zhang, Zhaoxiang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Fan, Junsong]'s Articles
[Zhang, Zhaoxiang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.