Prior knowledge is any information obtained from previous experience or data, and it can be represented as initial structure of model and the range of parameters while building models. In this work, we mainly discuss how to incorporate different prior knowledge into neural networks for the building of models. First, functional networks are introduced and studied for the reason that they can reproduce some physical and engineering properties to corresponding structure in a natural way. Thereafter, we propose a constraint neural networks which aim at the associating of neural networks and partially known relationships, and then apply this model to a real-world problems Though many methods have been proposed for the learning of functional networks and constraint neural networks, they are mainly based on maximum-likelihood(ML) or maximum a posterior (MAP) schemes. Bayesian methodology has a number of virtues in comparison with ML and MAP, e.g. the prior knowledge can be easily incorporated in a coherent way and Bayesian approach prevents overfitting problem naturally. In this thesis, we mainly study the the properties of functional networks (mainly generalized associativity functional networks-GAFN) and constraint neural networks(mainly superposition case) under the Bayesian framework, and propose a variational Bayes method to approximate the posterior distribution over parameters of the functional networks and constraint neural networks. The main contributions of this thesis include following issues: 1．We propose a variational learning method to approximate the posterior distribution over parameters of the generalized associative functional networks under the Bayesian framework. By use of linear transformation, the equality constraints on the parameters are eliminated and the approximation to the posterior is restricted within a subspace of the parameters. Furthermore, a variational approximation for parameter with inequality constraints is also studied. 2．We bring forward generalized constraint neural networks for the purpose of associating neural networks with partially known relationships. Two typical cases - Superposition and Multiplication, are studied to discovery the basic characteristics of constraint neural networks. The identifiability of parameters and the singularity problem are introduced during the applications of this model. 3．Based on the generalized constraint neural networks, the parametric model and neural networks are combined to build the ecological model of the plant growth process. A lot of meaningful discussions have been made for the application of semi-parametric methods. We also developed a nonlinear function with respect to the threshold temperature, and applied this method to the parametric model in the constraint neural networks.
修改评论