Institutional Repository of Chinese Acad Sci, Inst Automat, CAS Key Lab Mol Imaging, Beijing 100190, Peoples R China
Differentiable RandAugment: Learning Selecting Weights and Magnitude Distributions of Image Transformations | |
Xiao, Anqi1; Shen, Biluo1; Tian, Jie1,2,3; Hu, Zhenhua1 | |
发表期刊 | IEEE TRANSACTIONS ON IMAGE PROCESSING |
ISSN | 1057-7149 |
2023 | |
卷号 | 32页码:2413-2427 |
通讯作者 | Tian, Jie(tian@ieee.org) |
摘要 | Automatic data augmentation is a technique to automatically search for strategies for image transformations, which can improve the performance of different vision tasks. RandAugment (RA), one of the most widely used automatic data augmentations, achieves great success in different scales of models and datasets. However, RA randomly selects transformations with equivalent probabilities and applies a single magnitude for all transformations, which is suboptimal for different models and datasets. In this paper, we develop Differentiable RandAugment (DRA) to learn selecting weights and magnitudes of transformations for RA. The magnitude of each transformation is modeled following a normal distribution with both learnable mean and standard deviation. We also introduce the gradient of transformations to reduce the bias in gradient estimation and KL divergence as part of the loss to reduce the optimization gap. Experiments on CIFAR-10/100 and ImageNet demonstrate the efficiency and effectiveness of DRA. Searching for only 0.95 GPU hours on ImageNet, DRA can reach a Top-1 accuracy of 78.19% with ResNet-50, which outperforms RA by 0.28% under the same settings. Transfer learning on object detection also demonstrates the power of DRA. The proposed DRA is one of the few that surpasses RA on ImageNet and has great potential to be integrated into modern training pipelines to achieve state-of-the-art performance. Our code will be made publicly available for out-of-the-box use. |
关键词 | Task analysis Training Data models Costs Optimization Search problems Upper bound Data augmentation automated machine learning differentiable optimization random augmentation |
DOI | 10.1109/TIP.2023.3265266 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China (NSFC)[92059207] ; National Natural Science Foundation of China (NSFC)[62027901] ; National Natural Science Foundation of China (NSFC)[81930053] ; National Natural Science Foundation of China (NSFC)[81227901] ; Chinese Academy of Sciences (CAS) Youth Interdisciplinary[JCTD-2021-08] ; Zhuhai High-Level Health Personnel Team Project[HLHPTP201703] ; Cloud TensorProcessing Unit (TPUs) from Google's TPU Research Cloud (TRC) |
项目资助者 | National Natural Science Foundation of China (NSFC) ; Chinese Academy of Sciences (CAS) Youth Interdisciplinary ; Zhuhai High-Level Health Personnel Team Project ; Cloud TensorProcessing Unit (TPUs) from Google's TPU Research Cloud (TRC) |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000981890900002 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/53264 |
专题 | 中国科学院分子影像重点实验室 |
通讯作者 | Tian, Jie |
作者单位 | 1.Chinese Acad Sci, Inst Automat, CAS Key Lab Mol Imaging, Beijing Key Lab Mol Imaging, Beijing 100190, Peoples R China 2.Beihang Univ, Beijing Adv Innovat Ctr Big Data Based Precis Med, Sch Engn Med, Beijing 100191, Peoples R China 3.Xidian Univ, Engn Res Ctr Mol & Neuro Imaging, Sch Life Sci & Technol, Minist Educ, Xian 710071, Peoples R China |
第一作者单位 | 中国科学院分子影像重点实验室 |
通讯作者单位 | 中国科学院分子影像重点实验室 |
推荐引用方式 GB/T 7714 | Xiao, Anqi,Shen, Biluo,Tian, Jie,et al. Differentiable RandAugment: Learning Selecting Weights and Magnitude Distributions of Image Transformations[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2023,32:2413-2427. |
APA | Xiao, Anqi,Shen, Biluo,Tian, Jie,&Hu, Zhenhua.(2023).Differentiable RandAugment: Learning Selecting Weights and Magnitude Distributions of Image Transformations.IEEE TRANSACTIONS ON IMAGE PROCESSING,32,2413-2427. |
MLA | Xiao, Anqi,et al."Differentiable RandAugment: Learning Selecting Weights and Magnitude Distributions of Image Transformations".IEEE TRANSACTIONS ON IMAGE PROCESSING 32(2023):2413-2427. |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论