Comprehensive Evaluation is a very important research issue in practice. However, it endures a lot of controversy since there exist no ground truths. From the viewpoint of machine learning, comprehensive evaluation is an unsupervised ranking problem. Unsupervised ranking faces two critical challenges. One is the evaluation of ranking models, independent of ranking labels. The other is how to determine the nonlinear complexity of ranking models. With the framework of “Data and Knowledge Driven” models, this thesis systematically discusses the evaluation of unsupervised ranking from multiattribute numerical observations of ranking candidates. The main contributions of this thesis are given as following: 1. Five essential ranking meta-rules, namely, Scale and Translation Invariance, Strict Monotonicity, Compatibility of Linearity and Nonlinearity, Smoothness, Explicitness of Parameter Size, are drawn formally from ranking domain knowledge. With all five essential meta-rules, a ranking rule can produce a ranking list as closely as possible to the latent ground-truth labels. All five meta-rules can also be a high level evaluation for ranking models and provide a comparable evaluation method, which is independent of ranking labels, for different unsupervised ranking models. 2. A ranking principal curve (RPC) model, which follows all the five essential ranking meta-rules, is presented for unsupervised ranking from multiattribute objects. RPC is the data skeleton passing through the middle of the data cloud and approximated by a strictly monotone curve. RPC provides a nonlinear “ranking coordinate”, instead of a linear ranking coordinate by the first principal component analysis. For application, RPC is parameterized by a cubic B′ezier curve with control points restricted in a hypercube. The RPC existence and the convergency of RPC learning algorithm are proved theoretically. With the presented RPC models, continuous scores are output for ranking candidates and provide more information than scores themselves. Moreover, RPC models have the capacity of linearity and nonlinearity which are determined by model parameters. The nonlinear complexity of RPC models are also determined by parameters. 3. For the importance ranking of attributes, a two-phase attribute selection algorithm is proposed based on the knowledge of strict monotonicity between attributes and grading scores for objects. Phase I removes those irrelevant attributes for ranking based on Spea...
修改评论