CASIA OpenIR  > 毕业生  > 博士学位论文
广义约束神经网络的约束施加方法研究及其在求解微分方程中的应用
曹林林1,2
学位类型工学博士
导师胡包钢 ; 赫然
2017-05-24
学位授予单位中国科学院研究生院
学位授予地点北京
关键词广义约束神经网络 局部性原理 先验信息 微分方程
其他摘要
      随着深度网络的提出,人工神经网络 (ANNs) 又一次取得了重大的研究 进展。 ANNs 已经应用到许多领域,比如智能感知,系统辨识和控制,决策 制定和数据挖掘。然而, ANNs 具有一个致命的缺陷,即“黑箱”特性。为 了增加 ANNs 的透明性, Hu 等人提出一种广义约束神经网络( Generalized Constraint Neural NetworkGCNN),该方法属于知识与数据共同驱动模型方法 框架
Knowledge-and-data-driven Modeling KDDM)
      在“知识”与“数据”共同驱动模型的框架下,基于广义约束神经网络, 本文从广义约束的约束施加问题出发,首次讨论了约束施加的局部性原理,并 提出广义约束神经网络求解微分方程的新思路。 论文的主要贡献总结如下:
1. 本文提出一种具有等式约束的广义约束神经网络回归模型( GCNN_EC)。 在知识与数据共同驱动的基本框架下,本文构建了一种新的神经网络 输出模型,该模型的输出包括两项,第一项是由传统神经网络的输出构 成(数据驱动子模型),第二部分是由等式约束条件构成(知识驱动子模 型),以上两项以加权求和的形式耦合而成,权重的定义由数据到约束空 间的距离构成。该方法将等式约束问题转化为无约束优化问题,并且使 用线性优化方法去求解,因此约束条件的凸性不再是一个必要条件。与 已有方法相比, GCNN_EC 方法具有三个显著特点:第一, GCNN_EC 法可以连续地、严格地满足等式约束,而其他方法需要事先将等式约束 进行离散化;第二, GCNN_EC 可以处理含有未知参数的广义约束施加问 题,同时可以得到未知参数的合理估计值,而其他方法不能处理此类广 义约束施加问题;第三,该模型的算法复杂度和使用传统神经网络的复 杂度相同。
2. 针对人类大脑记忆的局部性原理,对等式约束施加方法进行了首次探 讨。局部性原理是大脑获得高效计算与节约能耗的重要机制。本文考察RBFNN+Lagrange multiplier, BVC-RBFGCNN + Lagrange interpolationGCNN_EC 方法的约束施加方式。本文首次从“功能信号”和“修正
信号”两个角度考察了本文方法与 RBFNN+Lagrange multiplier 的局部性 与全局性,通过图示揭示了本文方法遵循人类大脑记忆的局部性准则。 传统拉格朗日乘子法等解约束优化问题关注的是寻找满足约束的最优值, 从未对约束的施加方式进行讨论,本文重点讨论了约束施加问题中的局 部性原理。本文从知识与数据耦合方式的角度提出一个开放式问题:拉 格朗日方法是全局施加方法还是局部施加方法?大脑是否应用了拉格朗 日方法?本文对约束施加方法的研究有助于对该问题的理解和回答。
3. 开展了 GCNN_EC 对常微分方程和偏微分方程中的系统性数值统计对比 实验,对比的方法包括现有的其他神经元网络方法。数值实验验证了新 方法的各方面优越性。首先,当边界条件包含有未知参数时,即边界条 件以广义约束的形式给出时,传统方法不能处理此类约束,而本文方法 给出的学习算法不仅能求解方程,同时能够给出未知参数的合理估计值。 同时,该求解过程无额外的计算成本。第二,该方法可以完全满足等式 约束条件,包括插值点约束条件、 Dirichlet 边界条件和部分 Neumann 界条件(当边界条件具有显式的积分形式时),在函数回归性能和解微分 方程时具有更高的精度。 GCNN_EC 模型方法为提高模型的透明度提供 了一个可行而有效的路径。
;
    Artificial neural networks (ANNs) are one of the most popular models in artificial intelligence in the past decades. It has been widely applied to many different fields, such as system identification and control, decision making and data mining. However, the traditional ANNs suffer from an inherent limitation, i.e., the black box” characteristics. For adding transparency to ANNs, Hu et al. proposed a generalized constraint neural network (GCNN) approach. It can also be viewed as a knowledgeand-data-driven modeling (KDDM) approach.
    This work is based on the framework of GCNN and KDDM. Starting from the constraints imposing problems, this thesis firstly discusses the locality principle for imposing scheme of equality constraints . It also puts forward a new method to solve the differential equation using GCNN. The main contributions of this thesis are given as following:
1. This work proposes a generalized constraint neural network with equality constraints(GCNN_EC). Under the basic framework of knowledge-and-data-driven modeling approach, we construct a new prediction model with neural network. GCNN_EC is realized by a weighted combination of the output of the conventional radial basis function neural network (RBFNN) and the output expressed by the constraints. The weight is defined by the distance from data to the constraint space. GCNN_EC transfers equality constraint problems into unconstrained ones and solves them by a linear approach, so that convexity of constraints is no more an issue. Three advantages are obtained compared with existing methods. Firstly, GCNN_EC can satisfy equality constraint continuously and strictly unlike other methods which require discretization beforehand. Secondly, GCNN_EC can apply the generalized constraint with unknown parameters and at the same time get a reasonable estimate of the unknown parameters. Other methods can not handle such generalized constraint. Thirdly, GCNN_EC has the same algorithm complexity as traditional neural network. 

2. Inspired by the locality principle of human brain, it is an attempt for the first time to discuss the equality constraint imposing method. Locality principle is an important mechanism for the brain gaining efficient calculation and reducing energy consumption. This paper examines the constraint imposing scheme of RBFNN+Lagrange multiplier, BVC-RBF, GCNN + Lagrange interpolation and GCNN_EC. It is the first attempt to demonstrate the Locally Imposing Scheme (LIS) and Globally Imposing Scheme (GIS) of RBFNN+Lagrange multiplier and GCNN_EC from the view of “signal function” and “weight changes”. It provides graphical interpretations about the differences between GIS and LIS and reveals that the GCNN_EC obeys the locality principle of human brain memory. Traditional Lagrangian multiplier method focuses on finding the optimal value of constrained optimization problems and never discusses constraint imposing scheme. This paper focuses on the locality principle in the constraint imposing problem. This paper puts forward an open question from the point of coupling of knowledge and data: how to discover Lagrange multiplier method to be GIS or LIS? Does human brain apply the Lagrange multiplier method? The research of the constraint imposing scheme contributes to the understanding of the problem and answer.
3. Numerical comparison is conducted for GCNN_EC for solving ordinary differential equations and partial differential equations. GCNN_EC is demonstrated with other existing neural network methods. Numerical experiments verify superiority of the proposed method. First of all, when the boundary condition is expressed with unknown parameters, that is, in the form of generalized constraint, traditional methods can’t handle these constraints. GCNN_EC can not only solve the differential equation but also give a reasonable estimate for the
unknown parameters. At the same time, the solving process doesn’t need additional computational cost. Secondly, GCNN_EC can completely satisfy the equality constraints, including interpolation point constraints, Dirichlet boundary condition and some Neumann boundary condition (when boundary condition has explicit integral form). GCNN_EC has higher accuracy in obtaining function regression and solving differential equation. GCNN_EC model provides a 
feasible and effective path for improving the transparency of GCNN.
文献类型学位论文
条目标识符http://ir.ia.ac.cn/handle/173211/14794
专题毕业生_博士学位论文
作者单位1.中国科学院大学
2.中国科学院自动化研究所
推荐引用方式
GB/T 7714
曹林林. 广义约束神经网络的约束施加方法研究及其在求解微分方程中的应用[D]. 北京. 中国科学院研究生院,2017.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
thesis_cao.pdf(4115KB)学位论文 暂不开放CC BY-NC-SA请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[曹林林]的文章
百度学术
百度学术中相似的文章
[曹林林]的文章
必应学术
必应学术中相似的文章
[曹林林]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。