NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning
Peijie Yin; Lu Qi; Xuanyang Xi; Bo Zhang; Hong Qiao
2016
会议名称2016 International Joint Conference on Neural Networks (IJCNN)
会议录名称2016 International Joint Conference on Neural Networks (IJCNN)
会议日期24-29 July 2016
会议地点Vancouver, BC, Canada
摘要Generalization ability is widely acknowledged as one of the most important criteria to evaluate thequality of unsupervised models. The objective of our research is to find a better dropout method toimprove the generalization ability of convolutional deep belief network (CDBN), an unsupervised learningmodel for vision tasks. In this paper, the phenomenon of low feature diversity during the training process is investigated. The attention mechanism of human visual system is more focused on rare events and depresses well-known facts. Inspired by this mechanism, No Feature Left Behind Dropout (NFLBDropout), an adaptive dropout method is firstly proposed to automatically adjust the dropout rate feature-wisely. In the proposed method, the algorithm drops well-trained features and keeps poorly-trained ones with a high probability during training iterations. In addition, we apply two approximations of the quality of features, which are inspired by theory of saliency and optimization. Compared with themodel trained by standard dropout, experiment results show that our NFLB Dropout method improves not only the accuracy but the convergence speed as well.
关键词None
收录类别EI
文献类型会议论文
条目标识符http://ir.ia.ac.cn/handle/173211/12829
专题复杂系统管理与控制国家重点实验室_机器人理论与应用
通讯作者Peijie Yin
推荐引用方式
GB/T 7714
Peijie Yin,Lu Qi,Xuanyang Xi,et al. NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning[C],2016.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
NFLB Dropout_Improve(384KB)会议论文 开放获取CC BY-NC-SA浏览 请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Peijie Yin]的文章
[Lu Qi]的文章
[Xuanyang Xi]的文章
百度学术
百度学术中相似的文章
[Peijie Yin]的文章
[Lu Qi]的文章
[Xuanyang Xi]的文章
必应学术
必应学术中相似的文章
[Peijie Yin]的文章
[Lu Qi]的文章
[Xuanyang Xi]的文章
相关权益政策
暂无数据
收藏/分享
文件名: NFLB Dropout_Improve Generalization Ability by Dropping out the Best - A Biologically Inspired Adaptive Dropout Method For Unsupervised Learning.pdf
格式: Adobe PDF
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。