Stochastic Conjugate Gradient Algorithm With Variance Reduction

doi:10.1109/TNNLS.2018.2868835

CASIA OpenIR > 多模态人工智能系统全国重点实验室 > 模式分析与学习

	Stochastic Conjugate Gradient Algorithm With Variance Reduction
	Jin, Xiao-Bo 1; Zhang, Xu-Yao2 ; Huang, Kaizhu 3; Geng, Guang-Gang 4
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
	2019-05-01
卷号	30 期号:5 页码:1360-1369
通讯作者	Geng, Guang-Gang(gengguanggang@cnnic.cn)
摘要	Conjugate gradient (CG) methods are a class of important methods for solving linear equations and nonlinear optimization problems. In this paper, we propose a new stochastic CG algorithm with variance reduction(1) and we prove its linear convergence with the Fletcher and Reeves method for strongly convex and smooth functions. We experimentally demonstrate that the CG with variance reduction algorithm converges faster than its counterparts for four learning models, which may be convex, nonconvex or nonsmooth. In addition, its area under the curve performance on six large-scale data sets is comparable to that of the LIBLINEAR solver for the L2-regularized L2-loss but with a significant improvement in computational efficiency.
关键词	Computational efficiency covariance reduction empirical risk minimization (ERM) linear convergence stochastic conjugate gradient (CG)
DOI	10.1109/TNNLS.2018.2868835
关键词[WOS]	GLOBAL CONVERGENCE
收录类别	SCI
语种	英语
资助项目	National Basic Research Program of China[2012CB316301] ; National Key Research & Development Program[2016YFD0400104-5] ; National Natural Science Foundation of China[U1804326] ; National Natural Science Foundation of China[61473236] ; National Natural Science Foundation of China[61375039] ; National Natural Science Foundation of China[61602154] ; National Natural Science Foundation of China[61103138] ; Fundamental Research Funds for the Henan Provincial Colleges and Universities in Henan University of Technology[2016RCJH06] ; Fundamental Research Funds for the Henan Provincial Colleges and Universities in Henan University of Technology[2016RCJH06] ; National Natural Science Foundation of China[61103138] ; National Natural Science Foundation of China[61602154] ; National Natural Science Foundation of China[61375039] ; National Natural Science Foundation of China[61473236] ; National Natural Science Foundation of China[U1804326] ; National Key Research & Development Program[2016YFD0400104-5] ; National Basic Research Program of China[2012CB316301]
项目资助者	Fundamental Research Funds for the Henan Provincial Colleges and Universities in Henan University of Technology ; National Natural Science Foundation of China ; National Key Research & Development Program ; National Basic Research Program of China
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000466192100007
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：31[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://ir.ia.ac.cn/handle/173211/24224
专题	多模态人工智能系统全国重点实验室_模式分析与学习
通讯作者	Geng, Guang-Gang
作者单位	1.Henan Univ Technol, Dept Informat Sci & Engn, Zhengzhou 450001, Henan, Peoples R China 2.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 3.Xian Jiaotong Liverpool Univ, Dept Elect & Elect Engn, Suzhou 215123, Peoples R China 4.Chinese Acad Sci, Comp Network Informat Ctr, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Jin, Xiao-Bo,Zhang, Xu-Yao,Huang, Kaizhu,et al. Stochastic Conjugate Gradient Algorithm With Variance Reduction[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2019,30(5):1360-1369.
APA	Jin, Xiao-Bo,Zhang, Xu-Yao,Huang, Kaizhu,&Geng, Guang-Gang.(2019).Stochastic Conjugate Gradient Algorithm With Variance Reduction.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,30(5),1360-1369.
MLA	Jin, Xiao-Bo,et al."Stochastic Conjugate Gradient Algorithm With Variance Reduction".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 30.5(2019):1360-1369.