CASIA OpenIR  > 毕业生  > 博士学位论文
凯莱-克莱因度量学习
毕琰虹1,2
学位类型工学博士
导师吴福朝
2018-05-29
学位授予单位中国科学院大学
学位授予地点北京
关键词机器学习 度量学习 非欧几何度量 凯莱-克莱因度量
摘要
    对于计算机视觉和模式识别的许多应用问题,如图像识别、检索和聚类等,距离度量学习都扮演着至关重要的角色。目前,度量学习算法可分为线性和非线性两类:线性算法的基本原理是在所有可能的马氏度量中寻找合适的度量使目标损失函数最小;非线性算法则通过核技巧、流形学习、深度学习等方法获得更为一般化的距离度量。本文基于凯莱-克莱因度量,提出一种非线性度量学习算法,以提升传统度量学习对复杂分布数据的学习性能。主要工作包括以下几方面:
    1. 根据样本数据的统计特征(均值、协方差矩阵)和凯莱-克莱因度量的定义,构造了一种广义马氏度量,并指出了广义马氏度量与马氏度量之间的关系。在机器学习数据库中的性能测试表明:广义马氏度量比欧氏度量、传统的马氏度量有更好的分类性能。
    2. 提出一种基于MMC(Mahalanobis Metric Learning for Clustering,聚类马氏度量学习)准则的凯莱-克莱因度量学习算法(CK-MMC)。应用MMC的学习准则得到一个凯莱-克莱因度量,在最小化相似样本之间的凯莱-克莱因度量距离的同时最大化不相似样本之间的凯莱-克莱因度量距离。在CK-MMC问题中,使用梯度上升算法进行求解,实验结果表明CK-MMC有效地提升了传统MMC度量学习算法的分类性能。
    3. 提出一种基于LMNN(Large Margin Nearest Neighbors,最大间隔近邻)准则的凯莱-克莱因度量学习算法(CK-LMNN)。CK-LMNN使用相对距离约束学习得到一个凯莱-克莱因度量,使得每个样本与它的k-近邻同类,同时与不同类的样本点保持较大的距离。CK-LMNN问题使用梯度下降算法求解,实验结果表明CK-LMNN有效地提升了传统LMNN的分类性能。
    4. 提出一种多凯莱-克莱因度量学习算法(MCKML)。由同类样本集构造一个局部凯莱-克莱因度量,再将所有局部度量进行线性组合得到样本空间上的度量,称为多凯莱-克莱因度量。由于凯莱-克莱因度量内在的非线性,多凯莱-克莱因度量可以更好地模拟数据空间的非线性结构。在MCKML中,使用局部度量和组合系数交替优化的方式求解,实验结果表明使用多凯莱-克莱因度量有效提升了单个凯莱-克莱因度量学习算法的分类性能。
    5. 提出一种自适应约束凯莱-克莱因度量学习算法(CKseML)。在原始凯莱-克莱因度量学习算法的目标函数中,使用固定边界生成约束条件,对空间分布较为复杂的样本适应性不好。CKseML引入自适应收缩扩张约束,根据当前样本空间的分布状态,适时地收缩/扩张成对约束,以松弛边界约束条件,从而能适应分布更为复杂的样本空间。CKseML使用交替迭代方法优化凯莱-克莱因度量矩阵,实验结果表明CKseML比固定边界约束的凯莱-克莱因度量学习算法有更好的分类性能。
    本文首次将凯莱-克莱因度量引入距离度量学习领域,突破了现有马氏度量学习的框架,是一种特殊黎曼度量学习的实现,在计算机视觉和模式识别领域具有广泛的应用。
其他摘要
    Distance metric learning plays an important role in many computer vision and pattern recognition tasks, such as image recognition, retrieval and clustering. Existing research in metric learning includes linear and non-linear approaches. Linear metric learning methods aim to learn a proper Mahalanobis metric to minimize the given objective function. Non-linear metric learning methods obtain a more generalized metric by means of kernel trick, manifold learning and deep learning. In this paper, we present a non-linear metric learning framework based on Cayley-Klein metric, aiming to improve the performance of the traditional Mahalanobis metric learning method when dealing with complex data space. Our main contributions include:
    1. Based on the statistical properties of data space (mean and covariance matrix) and the definition of Cayley-Klein metric, we propose a special form of Cayley-Klein metric called generalized Mahalanobis metric and discuss its relationship with Mahalanobis metric. Experimental results show that the generalized Mahalanobis metric outperforms both Euclidean and traditional Mahalanobis metrics in classification task.
    2. We propose a Cayley-Klein metric learning method (CK-MMC) based on MMC (Mahalanobis Metric Learning for Clustering). CK-MMC aims to minimize distances between similar pairs and simultaneously maximize distances between dissimilar pairs. The optimization problem can be solved by the gradient ascent algorithm. Experiments have shown the superiority of CK-MMC over the traditional MMC metric learning method.
    3. We propose a Cayley-Klein metric learning method (CK-LMNN) based on LMNN (Large Margin Nearest Neighbors). CK-LMNN utilizes triplet-wise constraints to learn a Cayley-Klein metric, aiming to make the k-nearest neighbors of a data point lie in the same class as the data point, and meanwhile make data points from different classes are separated by a large margin. The optimization problem can be solved by the gradient descent algorithm. Experiments have shown the superiority of CK-LMNN over the traditional LMNN metric learning method.
    4. We propose a multiple Cayley-Klein metric learning method (MCKML). Multiple Cayley-Klein metric is defined as a linear combination of several local Cayley-Klein metrics, which are learned by data points from the same class. Since Cayley-Klein metric is a kind of non-linear metric, its combination could model the data space better, thus lead to an improved performance. MCKML is learned by iterative optimization over single Cayley-Klein metrics and their combination coefficients. Experiments on several benchmarks show that MCKML outperforms the single Cayley-Klein metric learning method.
    5. We propose an adaptive constraints based Cayley-Klein metric learning method (CKseML). The original Cayley-Klein metric learning methods use conventional pairwise and triplet-wise constraints, which are fixed bound based constraints, may not perform well when the data distribution become complex. CKseML adaptively shrinks/expands the pairwise constraints considering the current data distribution, leading to better adaptability for complex data distribution. Experimental results demonstrate that CKseML achieves better performance than the fixed constraints based Cayley-Klein metric learning methods.
    This paper first introduces the Cayley-Klein metric in the community of metric learning. As a special kind of Riemannian metric learning, it is a powerful alternative to the traditional Mahalanobis metric and can be widely used in computer vision and pattern recognition.
语种中文
文献类型学位论文
条目标识符http://ir.ia.ac.cn/handle/173211/21070
专题毕业生_博士学位论文
作者单位1.中国科学院自动化研究所
2.中国科学院大学
推荐引用方式
GB/T 7714
毕琰虹. 凯莱-克莱因度量学习[D]. 北京. 中国科学院大学,2018.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
Thesis_byh.pdf(4575KB)学位论文 暂不开放CC BY-NC-SA请求全文
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[毕琰虹]的文章
百度学术
百度学术中相似的文章
[毕琰虹]的文章
必应学术
必应学术中相似的文章
[毕琰虹]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。