NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning
Peijie Yin; Lu Qi; Xuanyang Xi; Bo Zhang; Hong Qiao
2016
Conference Name2016 International Joint Conference on Neural Networks (IJCNN)
Source Publication2016 International Joint Conference on Neural Networks (IJCNN)
Conference Date24-29 July 2016
Conference PlaceVancouver, BC, Canada
AbstractGeneralization ability is widely acknowledged as one of the most important criteria to evaluate thequality of unsupervised models. The objective of our research is to find a better dropout method toimprove the generalization ability of convolutional deep belief network (CDBN), an unsupervised learningmodel for vision tasks. In this paper, the phenomenon of low feature diversity during the training process is investigated. The attention mechanism of human visual system is more focused on rare events and depresses well-known facts. Inspired by this mechanism, No Feature Left Behind Dropout (NFLBDropout), an adaptive dropout method is firstly proposed to automatically adjust the dropout rate feature-wisely. In the proposed method, the algorithm drops well-trained features and keeps poorly-trained ones with a high probability during training iterations. In addition, we apply two approximations of the quality of features, which are inspired by theory of saliency and optimization. Compared with themodel trained by standard dropout, experiment results show that our NFLB Dropout method improves not only the accuracy but the convergence speed as well.
KeywordNone
Indexed ByEI
Document Type会议论文
Identifierhttp://ir.ia.ac.cn/handle/173211/12829
Collection复杂系统管理与控制国家重点实验室_机器人理论与应用
Corresponding AuthorPeijie Yin
Recommended Citation
GB/T 7714
Peijie Yin,Lu Qi,Xuanyang Xi,et al. NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning[C],2016.
Files in This Item: Download All
File Name/Size DocType Version Access License
NFLB Dropout_Improve(384KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Peijie Yin]'s Articles
[Lu Qi]'s Articles
[Xuanyang Xi]'s Articles
Baidu academic
Similar articles in Baidu academic
[Peijie Yin]'s Articles
[Lu Qi]'s Articles
[Xuanyang Xi]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Peijie Yin]'s Articles
[Lu Qi]'s Articles
[Xuanyang Xi]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: NFLB Dropout_Improve Generalization Ability by Dropping out the Best - A Biologically Inspired Adaptive Dropout Method For Unsupervised Learning.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.