Distribution Unified and Probability Space Aligned Teacher-Student Learning for Imbalanced Visual Recognition | |
Zhang, Shaoyu1,2![]() ![]() ![]() ![]() | |
发表期刊 | IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
![]() |
ISSN | 1051-8215 |
2024-04-01 | |
卷号 | 34期号:4页码:2414-2425 |
摘要 | Imbalanced label distribution is usually the case for real-world data, which poses a challenge for training unbiased recognition model. In this paper, we study two underlying mismatches, i.e., distribution mismatch and probability space mismatch, present in class-imbalanced learning. Firstly, we analyze the label distribution mismatch between imbalanced training data and balanced test data, and introduce a distribution unified framework to unify the two distributions through probability conversion. Secondly, we analyze that the utilization of cross-entropy loss under the proposed framework may lead to probability space mismatch, where the conversion of the predictive probability is implemented in softmax probability space while the comparison with one-hot label is implemented in true probability space. To alleviate this dilemma, we involve a teacher model and formulate a teacher-student learning strategy, which contains two novel techniques. The Teacher Guided Label Smoothing (TGLS) is first proposed to relax the one-hot label to smoother pseudo softmax probability, which is more aligned with the softmax probability space. Additionally, we propose Distribution Unified Knowledge Distillation (DU-KD) under the proposed framework to further reduce both the mismatches. Experiments on several benchmarks confirm the top-level performance of the proposed method. |
关键词 | Class-imbalanced learning distribution mismatch probability space mismatch teacher-student learning |
DOI | 10.1109/TCSVT.2023.3311142 |
收录类别 | SCI |
语种 | 英语 |
WOS研究方向 | Engineering |
WOS类目 | Engineering, Electrical & Electronic |
WOS记录号 | WOS:001197960500075 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | 目标检测、跟踪与识别 |
国重实验室规划方向分类 | 小样本高噪声数据学习 |
是否有论文关联数据集需要存交 | 否 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/57042 |
专题 | 智能制造技术与系统研究中心_多维数据分析(彭思龙)-技术团队 |
通讯作者 | Chen, Chen |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China 3.CNPC, GWDC Mud Logging Co, Panjin 124010, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhang, Shaoyu,Chen, Chen,Xie, Qiong,et al. Distribution Unified and Probability Space Aligned Teacher-Student Learning for Imbalanced Visual Recognition[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2024,34(4):2414-2425. |
APA | Zhang, Shaoyu,Chen, Chen,Xie, Qiong,Sun, Haigang,Dong, Fei,&Peng, Silong.(2024).Distribution Unified and Probability Space Aligned Teacher-Student Learning for Imbalanced Visual Recognition.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,34(4),2414-2425. |
MLA | Zhang, Shaoyu,et al."Distribution Unified and Probability Space Aligned Teacher-Student Learning for Imbalanced Visual Recognition".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 34.4(2024):2414-2425. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Distribution_Unified(2511KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论