Cyclic Differentiable Architecture Search | |
Yu HY(俞宏远)1,2![]() ![]() ![]() | |
发表期刊 | IEEE Transactions on Pattern Analysis and Machine Intelligence
![]() |
2022 | |
卷号 | 2022期号:TPAMI.2022.3153065页码:DOI 10.1109 |
文章类型 | 计算机视觉 |
摘要 | Differentiable ARchiTecture Search, i.e., DARTS, has drawn great attention in neural architecture search. It tries to find the optimal architecture in a shallow search network and then measure its performance in a deep evaluation network. The independent optimization of the search and evaluation networks, however, leaves a room for potential improvement by allowing interaction between the two networks. To address the problematic optimization issue, we propose new joint optimization objectives and a novel Cyclic Differentiable ARchiTecture Search framework, dubbed CyDAS. Considering the structure difference, CyDAS builds a cyclic feedback mechanism between the search and evaluation networks with introspective distillation. First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized. Second, the architecture weights in the search network are further optimized by the label supervision in classification, as well as the regularization from the evaluation network through feature distillation. Repeating the above cycle results in a joint optimization of the search and evaluation networks and thus enables the evolution of the architecture to fit the final evaluation network. The experiments and analysis on CIFAR, ImageNet and NAS-Bench-201 demonstrate the effectiveness of the proposed approach over the state-of-the-art ones. Specifically, in the DARTS search space, we achieve 97.52% top-1 accuracy on CIFAR10 and 76.3% top-1 accuracy on ImageNet. In the chain-structured search space, we achieve 78.2% top-1 accuracy on ImageNet, which is 1.1% higher than EfficientNet-B0. Our code and models are publicly available at https://github.com/researchmm/CyDAS.git. |
语种 | 英语 |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/48518 |
专题 | 模式识别实验室 |
通讯作者 | Wang L(王亮) |
作者单位 | 1.中国科学院大学 2.中国科学院自动化研究所 3.Microsoft Research Asia 4.the Department of Computer Science, Stony Brook University |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Yu HY,Peng HW,Huang Y,et al. Cyclic Differentiable Architecture Search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,2022(TPAMI.2022.3153065):DOI 10.1109. |
APA | Yu HY.,Peng HW.,Huang Y.,Fu JL.,Du, Hao.,...&Lin, Haibin.(2022).Cyclic Differentiable Architecture Search.IEEE Transactions on Pattern Analysis and Machine Intelligence,2022(TPAMI.2022.3153065),DOI 10.1109. |
MLA | Yu HY,et al."Cyclic Differentiable Architecture Search".IEEE Transactions on Pattern Analysis and Machine Intelligence 2022.TPAMI.2022.3153065(2022):DOI 10.1109. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Cyclic_Differentiabl(5244KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论