人脸识别论文：Neighborhood Components Analysis.pdfVIP

Neighbourhood Components Analysis Jacob Goldberger, Sam Roweis, Geoff Hinton, Ruslan Salakhutdinov Department of Computer Science, University of Toronto {jacob,roweis,hinton,rsalakhu }@cs.toronto.edu Abstract In this paper we propose a novel method for learning a Mahalanobis distance measure to be used in the KNN classification algorithm. The algorithm directly maximizes a stochastic variant of the leave-one-out KNN score on the training set. It can also learn a low-dimensional lin- ear embedding of labeled data that can be used for data visualization and fast classification. Unlike other methods, our classification model is non-parametric, making no assumptions about the shape of the class distributions or the boundaries between them. The performance of the method is demonstrated on several data sets, both for metric learning and linear dimensionality reduction. 1 Introduction Nearest neighbor (KNN) is an extremely simple yet surprisingly effective method for clas- sification. Its appeal stems from the fact that its decision surfaces are nonlinear, there is only a single integer parameter (which is easily tuned with cross-validation), and the expected quality of predictions improves automatically as the amount of training data in- creases. These advantages, shared by many non-parametric methods, reflect the fact that although the final classification machine has quite high capacity (since it accesses the entire reservoir of training data at test time), the trivial learning procedure rarely causes overfitting itself. However, KNN suffers from two very serious drawbacks. The first is computational, since it must store and search through the entire tra