Knowledge Commons of Institute of Automation,CAS
Bayesian Automatic Model Compression | |
Wang, Jiaxing1,2![]() ![]() ![]() | |
发表期刊 | IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING
![]() |
ISSN | 1932-4553 |
2020-05-01 | |
卷号 | 14期号:4页码:727-736 |
摘要 | Model compression has drawn great attention in deep learning community. A core problem in model compression is to determine the layer-wise optimal compression policy, e.g., the layer-wise bit-width in network quantization. Conventional hand-crafted heuristics rely on human experts and are usually sub-optimal, while recent reinforcement learning based approaches can be inefficient during the exploration of the policy space. In this article, we propose Bayesian automatic model compression (BAMC), which leverages non-parametric Bayesian methods to learn the optimal quantization bit-width for each layer of the network. BAMC is trained in a one-shot manner, avoiding the back and forth (re)-training in reinforcement learning based approaches. Experimental results on various datasets validate that our proposed methods can find reasonable quantization policies efficiently with little accuracy drop for the quantized network. |
关键词 | Quantization automatic machine learning model compression Bayes methods Mathematical model Training Mixture models Optimization Machine learning Bayesian learning quantizartion explainability |
DOI | 10.1109/JSTSP.2020.2977090 |
收录类别 | SCI |
语种 | 英语 |
资助项目 | National Natural Science Foundation of China[61876182] ; National Natural Science Foundation of China[61906193] ; National Natural Science Foundation of China[61906195] ; Strategic Priority Research Program of Chinese Academy of Science[XDB32050200] ; Advance Research Program[31511130301] |
项目资助者 | National Natural Science Foundation of China ; Strategic Priority Research Program of Chinese Academy of Science ; Advance Research Program |
WOS研究方向 | Engineering |
WOS类目 | Engineering, Electrical & Electronic |
WOS记录号 | WOS:000565858900011 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
七大方向——子方向分类 | AI芯片与智能计算 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/41514 |
专题 | 复杂系统认知与决策实验室_高效智能计算与学习 |
通讯作者 | Cheng, Jian |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100190, Peoples R China 3.Chinese Univ Hong Kong, Dept Comp Sci & Engn, Hong Kong 999077, Peoples R China 4.Tencent AI Lab, Machine Learning Grp, Shenzhen 518000, Peoples R China 5.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 6.CAS Ctr Excellence Brain Sci & Intelligence Techn, Beijing 100190, Peoples R China |
第一作者单位 | 中国科学院自动化研究所 |
通讯作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Wang, Jiaxing,Bai, Haoli,Wu, Jiaxiang,et al. Bayesian Automatic Model Compression[J]. IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING,2020,14(4):727-736. |
APA | Wang, Jiaxing,Bai, Haoli,Wu, Jiaxiang,&Cheng, Jian.(2020).Bayesian Automatic Model Compression.IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING,14(4),727-736. |
MLA | Wang, Jiaxing,et al."Bayesian Automatic Model Compression".IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 14.4(2020):727-736. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
Bayesian Automatic M(1918KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论