CASIA OpenIR  > 毕业生  > 博士学位论文
Alternative TitleRBF Neural Networks Regression Model Subject to Linear Priors
Thesis Advisor胡包钢
Degree Grantor中国科学院研究生院
Place of Conferral中国科学院自动化研究所
Degree Discipline计算机应用技术
Keyword透明度 径向基函数神经网络 先验信息 线性先验 线性约束 消元法 Transparency Rbf Networks Prior Information Linear Priors Linear Constraints Elimination Method
Abstract人工神经网络是一个优越的通用逼近器,且得到了很好的发展和应用。但是由于存在一些缺陷,如“黑箱”特性和忽视存在的先验信息,神经网络的发展和应用受到了很大程度上的限制。而在实际问题中,将先验信息嵌入到真实模型的设计中是非常有价值的。尤其是在先验信息容易得到的某些真实的特定问题下,充分的利用先验能够很好地克服训练数据的缺陷而改善模型的性能。因此,在本篇论文中,我们主要考虑将先验信息嵌入到模型中去增加神经网络的透明度和改善模型的性能。本篇论文的贡献主要包括以下几个部分: 1. 基于排序先验的径向基函数神经网络的研究. 对比硬性约束的方法,我们提出一种较为合理的方式处理排序先验, 即信息检索评价准则(Information Retrieval,IR):最大化归一化折扣累计增益(Normalized Discounted Cumulative Gain,NDCG)。另外, 本文还揭示了成对损失(Pairwise Loss)和NDCG的关系,并得到了加权的成对损失是(1 −NDCG)的一个上界的结论。另外,一些真实回归问题的数值实验表明此算法的有效性,且在训练样本缺失或含有噪声的情况下,模型性能的改善尤为明显。 2. 基于一类线性先验的广义约束神经网络的研究. 目前,存在的一些算法都是针对特定的先验信息而提出的。因此,本文主要希望发展一种通用的方法,称为带广义约束的神经网络(Generalized Constraints Neural Networks - Linear Priors,GCNN-LP), 处理一类线性先验信息,比如排序先验,界先验,单调性先验等。相关贡献包括: - 提出一种结构模式显式地嵌入线性先验,比单纯的算法模式更能增加神经网络的透明度; - 利用柔性约束处理不正确或噪声的先验信息,比硬性约束更能适应噪声的信息; - 提出一种直接的消元法和最小二乘相结合的算法统一处理硬性和柔性约束,比拉格朗日方法具有更好的精度和时间复杂度; - 提出一种修正的GCNN-LP去处理(高维)连续约束的先验信息,比如界先验,单调性先验等。 一些人工合成数据和真实数据集下的实验证明了GCNN-LP算法的有效性。 3. 基于GCNN-LP工具箱的开发. 为了GCNN-LP算法的发展和应用,本文基于Scilab开源平台开发了非线性回归模型的工具箱。不同于其他的工具箱,此工具箱主要是给出一个嵌入先验信息的框架性和通用性的方法。为了有助于理解GCNN-LP算法和使用工具箱的方便,工具箱中提供了众多的实例来说明用户可以通过对GCNN-LP算法源码的简单修改而得到适合不同先验或问题方法。
Other AbstractBecause of the ability to provide excellent universal function approximation for multivariate input/output spaces on a training set, artificial neural networks are widely used in various applications. However, some disadvantages such as “Black-box” characteristics, ignoring most of existing prior information, greatly restrain further advances of the neural networks technique in applications. In real-word problems, prior information is valuable to be incorporated into a practical learning of models. Making the best use of information can overcome the limitation of the training data and improve the performance of the networks, especially in some real-word domains where extensive prior information is available. Therefore, in this thesis, the incorporation of prior information in modelings is our specific concern for adding transparency and improving the performance of the networks. The main contributions of this thesis are as follows: 1. Radial Basis Function (RBF) Neural Networks with Ranking Prior. Comparing with the method that treats the ranking information as hard constraints, we handle ranking reasonably by maximization of Normalized Discounted Cumulative Gain (NDCG) as Information Retrieval (IR) evaluation measure, which is used to evaluate the performance ranking model. In addition, a connection between weighted pairwise loss and NDCG is also revealed, and an upper bound of one minus NDCG is given by weighted pairwise loss. Numerical results from some existing benchmark regression problems confirm the beneficial aspects on the proposed approach. When training data are scarce or with noise, the improvement of the model will be better. 2. Generalized Constraint Neural Networks with Linear Priors. Techniques in the existing methods embedded prior vary from different prior information. Therefore, this part will concentrate on developing a more generic approach, called Generalized Constraint Neural Networks-Linear Priors (GCNN-LP), for handling a class of linear priors, such as ranking list, boundary, monotonicity, etc. The key contributions of this part include: - An explicitly structural mode, which may add a higher degree of transparency than using a pure algorithm mode, is proposed for embedding linear prior. - Soft constraints, which can handle prior information with noise better than hard constraints, are investigated. - Direct elimination and least squares approach, which produces better performances in both accuracy and computational cost o...
Other Identifier200918014629084
Document Type学位论文
Recommended Citation
GB/T 7714
瞿亚军. 基于线性先验的径向基函数神经网络的研究[D]. 中国科学院自动化研究所. 中国科学院研究生院,2012.
Files in This Item:
File Name/Size DocType Version Access License
CASIA_20091801462908(1819KB) 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[瞿亚军]'s Articles
Baidu academic
Similar articles in Baidu academic
[瞿亚军]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[瞿亚军]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.