CASIA OpenIR  > 模式识别国家重点实验室  > 多媒体计算与图形学
Learning with Average Top-k Loss
Fan, Yanbo1,3,4; Lyu, Siwei1; Ying, Yiming2; Hu, Bao-Gang3,4
Conference NameNeural Information Processing Systems (NIPS)
Conference Date2017
Conference PlaceLong Beach, CA, USA
In this work, we introduce the average top-k (ATk) loss as a new aggregate loss for supervised learning, which is the average over the k largest individual losses over a training dataset. We show that the ATk loss is a natural generalization of the two widely used aggregate losses, namely the average loss and the maximum loss, but can combine their advantages and mitigate their drawbacks to better adapt to different data distributions. Furthermore, it remains a convex function over all individual losses, which can lead to convex optimization problems that can be solved effectively with conventional gradient-based methods. We provide an intuitive interpretation of the ATk loss based on its equivalent effect on the continuous individual loss functions, suggesting that it can reduce the penalty on correctly classified data. We further give a learning theory analysis of MATk learning on the classification calibration of the ATk loss and the error bounds of ATk-SVM. We demonstrate the applicability of minimum average top-k learning for binary classification and regression using synthetic and real datasets.
KeywordSupervised Learning Aggregate Loss Average Top-k
Indexed ByEI
Document Type会议论文
Corresponding AuthorLyu, Siwei
Affiliation1.Department of Computer Science, University at Albany, SUNY
2.Department of Mathematics and Statistics, University at Albany, SUNY
3.National Laboratory of Pattern Recognition, CASIA
4.University of Chinese Academy of Sciences
First Author AffilicationInstitute of Automation, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
Fan, Yanbo,Lyu, Siwei,Ying, Yiming,et al. Learning with Average Top-k Loss[C],2017.
Files in This Item: Download All
File Name/Size DocType Version Access License
Learning with Averag(532KB)会议论文 开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Fan, Yanbo]'s Articles
[Lyu, Siwei]'s Articles
[Ying, Yiming]'s Articles
Baidu academic
Similar articles in Baidu academic
[Fan, Yanbo]'s Articles
[Lyu, Siwei]'s Articles
[Ying, Yiming]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Fan, Yanbo]'s Articles
[Lyu, Siwei]'s Articles
[Ying, Yiming]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: Learning with Average Top-k Loss.pdf
Format: Adobe PDF
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.