Knowledge Commons of Institute of Automation,CAS
Discriminative Feature Selection by Nonparametric Bayes Error Minimization | |
Yang, Shuang-Hong1; Hu, Bao-Gang2 | |
发表期刊 | IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING |
2012-08-01 | |
卷号 | 24期号:8页码:1422-1434 |
文章类型 | Article |
摘要 | Feature selection is fundamental to knowledge discovery from massive amount of high-dimensional data. In an effort to establish theoretical justification for feature selection algorithms, this paper presents a theoretically optimal criterion, namely, the discriminative optimal criterion (DoC) for feature selection. Compared with the existing representative optimal criterion (RoC, [21]) which retains maximum information for modeling the relationship between input and output variables, DoC is pragmatically advantageous because it attempts to directly maximize the classification accuracy and naturally reflects the Bayes error in the objective. To make DoC computationally tractable for practical tasks, we propose an algorithmic framework, which selects a subset of features by minimizing the Bayes error rate estimated by a nonparametric estimator. A set of existing algorithms as well as new ones can be derived naturally from this framework. As an example, we show that the Relief algorithm [20] greedily attempts to minimize the Bayes error estimated by the k-Nearest-Neighbor (kNN) method. This new interpretation insightfully reveals the secret behind the family of margin-based feature selection algorithms [28], [14] and also offers a principled way to establish new alternatives for performance improvement. In particular, by exploiting the proposed framework, we establish the Parzen-Relief (P-Relief) algorithm based on Parzen window estimator, and the MAP-Relief (M-Relief) which integrates label distribution into the max-margin objective to effectively handle imbalanced and multiclass data. Experiments on various benchmark data sets demonstrate the effectiveness of the proposed algorithms. |
关键词 | Feature Selection Discriminative Optimal Criterion Feature Weighting |
WOS标题词 | Science & Technology ; Technology |
关键词[WOS] | BHATTACHARYYA DISTANCE ; ALGORITHMS |
收录类别 | SCI |
语种 | 英语 |
WOS研究方向 | Computer Science ; Engineering |
WOS类目 | Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical & Electronic |
WOS记录号 | WOS:000305900500006 |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.ia.ac.cn/handle/173211/2845 |
专题 | 多模态人工智能系统全国重点实验室_多媒体计算 |
作者单位 | 1.Georgia Inst Technol, Coll Comp, Atlanta, GA 30318 USA 2.Chinese Acad Sci, NLPR, Beijing 100190, Peoples R China |
推荐引用方式 GB/T 7714 | Yang, Shuang-Hong,Hu, Bao-Gang. Discriminative Feature Selection by Nonparametric Bayes Error Minimization[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2012,24(8):1422-1434. |
APA | Yang, Shuang-Hong,&Hu, Bao-Gang.(2012).Discriminative Feature Selection by Nonparametric Bayes Error Minimization.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,24(8),1422-1434. |
MLA | Yang, Shuang-Hong,et al."Discriminative Feature Selection by Nonparametric Bayes Error Minimization".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 24.8(2012):1422-1434. |
条目包含的文件 | 下载所有文件 | |||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 | ||
yangsh12.pdf(1258KB) | 期刊论文 | 作者接受稿 | 开放获取 | CC BY-NC-SA | 浏览 下载 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论