Density estimation is a fundamental problem in pattern recognition and machine learning. It is particularly important for classification using the Bayes decision rule. The Gaussian Mixture Model (GMM) is a popular model for density estimation because of its great capability of approximating arbitrary distributions. The Expectation Maximization (EM) algorithm, based on maximum likelihood, is a basic approach for GMM parameter estimation. However, density estimation in high-dimensional data spaces is a challenge due to the sparseness of data which is well-known as "the curse of dimensionality". Reducing the dimensionality of features can overcome the curse of dimensionality, but how to combine dimensionality reduction with GMM is a concern. On the other hand, the GMM is a generative model, with parameters estimated for each class independently; without considering decision boundaries in training, the obtained models do not necessarily give high classification accuracy. While discriminative learning can improve the classification accuracy of the model. Aiming at these problems, this thesis studies model structure selection in high-dimensional space and discriminative learning for GMMs. The main contributions of this thesis are as follows. (1) We propose a Pooled Subspace Mixture Density (PSMD) model for classification, which represents the density in full space and estimate the common subspace and Gaussian mixture simultaneously under the EM framework. Each Gaussian component is represented as the product of an elliptical Gaussian in subspace and a spherical Gaussian in the complementary subspace. Firstly, the EM algorithm estimates the model parameters of full space, including the weighting coefficients, the means and the covariance matrix; Then we compute the pooled covariance and the pooled subspace, and project each Gaussian component into the pooled subspace. In the pooled subspace, each component is a Gaussian model, and the density in the complementary subspace is characterized by pooled eigenvalue. In order to improve the classification accuracy, the pooled eigenvalue is decided by cross validation. The experimental results on UCI datasets demonstrate that in most cases, the proposed method yields higher classification accuracies than the previous ones. (2) For the subspace GMM density model, we propose a discriminative learning method. The minimum classification error (MCE) criterion is chosen to optimize all the parameters by stochastic gradient...
修改评论