CASIA OpenIR  > 智能感知与计算研究中心
Information-Theoretic Measures for Objective Evaluation of Classifications
Baogang Hu; Ran He(赫然); Xiaotong Yuan
Source PublicationChinese Acta Automatica Sinica
AbstractThis work presents a systematic study of objective evaluations of abstaining classifications using Information-Theoretic Measures (ITMs). First, we define objective measures for which they do not depend on any free parameter. This defi- nition provides technical simplicity for examining “objectivity” or “subjectivity” directly to classification evaluations. Second, we propose twenty four normalized ITMs, derived from either mutual information, divergence, or crossentropy, for investigation. Contrary to conventional performance measures that apply empirical formulas based on users’ intuitions or preferences, the ITMs are theoretically more sound for realizing objective evaluations of classifications. We apply them to distinguish “error types” and “reject types” in binary classifications without the need for input data of cost terms. Third, to better understand and select the ITMs, we suggest three desirable features for classification assessment measures, which appear more crucial and appealing from the viewpoint of classification applications. Using these features as “meta-measures”, we can reveal the advantages and limitations of ITMs from a higher level of evaluation knowledge. Numerical examples are given to corroborate our claims and compare the differences among the proposed measures. The best measure is selected in terms of the meta-measures, and its specific properties regarding error types and reject types are analytically derived.
KeywordAbstaining Classifications Error Types Reject Types Entropy Similarity Objectivity
Document Type期刊论文
Recommended Citation
GB/T 7714
Baogang Hu,Ran He,Xiaotong Yuan. Information-Theoretic Measures for Objective Evaluation of Classifications[J]. Chinese Acta Automatica Sinica,2012,38(7):1169-1182.
APA Baogang Hu,Ran He,&Xiaotong Yuan.(2012).Information-Theoretic Measures for Objective Evaluation of Classifications.Chinese Acta Automatica Sinica,38(7),1169-1182.
MLA Baogang Hu,et al."Information-Theoretic Measures for Objective Evaluation of Classifications".Chinese Acta Automatica Sinica 38.7(2012):1169-1182.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Baogang Hu]'s Articles
[Ran He(赫然)]'s Articles
[Xiaotong Yuan]'s Articles
Baidu academic
Similar articles in Baidu academic
[Baogang Hu]'s Articles
[Ran He(赫然)]'s Articles
[Xiaotong Yuan]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Baogang Hu]'s Articles
[Ran He(赫然)]'s Articles
[Xiaotong Yuan]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.