Because of the parameter independent assumption, the traditional discriminative models cannot be influenced by the unlabeled data. Discriminative learning with less supervision tries to modify this assumption and/or exploits prior knowledge to use the unlabeled data to improve the performance of the models. How to extend the traditional discriminative models to less supervised problems has attracted more and more researchers. In this thesis, we give a definition of less supervised problem and review the status of the research in this topic. The main contributions of this thesis include three: 1.Semi-supervised Classification. Semi-supervised SVM just utilizes the instances lying in the marginregion and abandons the other geometry information contained in boththe other labeled and unlabeled points. We propose the Compact Assumption. Based on the assumption, we design the Compact Margin Machine (CMM) to embed the global information into the model and the optimization could be solved by Constrained Convex-Concave Procedure (CCCP). Experiments valid the classification ability of CMM. 2.Discriminative Clustering.By extending the Low-Density Separation Assumption to clustering problems, we implement this assumption for clustering via information-theoretic method and Bayesian nonparametrics. The key contributions of this part include: - We provide interpretation of several famous discriminative clustering algorithms from information-theoretic view. - Based on the unified information-theoretic framework, we propose several novel discriminative clustering algorithms, such as Logistics Clustering, Unsupervised CRF and Maximum Relative Margin Clustering (MRMC). We also accelerate the MRMC to make it practical. - We extend the supervised Bayesian nonparametrics, Gaussian Process, to unsupervised setting. By introducing the `margin' concept to this model, our algorithm could utilize Universum to guide clustering. Experimental results show that our methods are comparable, sometime even better than the extant algorithms. 3. Semi-supervised Metric Learning. We first extend the Maximum Entropy Principle to metric learning. Compared to the traditional metric learning algorithms that just involve 'must-link' and 'cannot-link' pairs as the input, we introduce Posterior Sparsity Assumption to realize the learning ability of discriminative models on unlabeled pairs. Experimental results illustrate our method is statistically better than existing algorithms, especially...
修改评论