Research Paper on Intrinsic Dimension - University of Minnesota 对内在维度-明尼苏达大学的研究.docVIP

Estimating Intrinsic Dimension by Justin Eberhardt Department of Mathematics and Statistics University of Minnesota Duluth Duluth, MN 55812 June 2007 Estimating Intrinsic Dimension A project submitted to the faculty of the Graduate School of the University of Minnesota by Justin Eberhardt In partial fulfillment of the requirements for the degree of Master of Science In Applied and Computational Mathematics June 2007 Estimating Intrinsic Dimension Abstract The intrinsic dimension of a dataset is often much less than the dimension of the original dataset. ?It is valuable to know the intrinsic dimension of a dataset so that the high dimensional dataset can be replaced by a lower dimensional dataset that is easier to manipulate. ?Traditional intrinsic dimension estimators, such as principal component analysis can only be used on linear spaces. Non-linear manifolds require other methods, such as nearest-neighbor estimators. ?We will compare three nearest-neighbor estimators based on several criteria and show that two estimators perform well on a wide range of non-linear datasets.  I would like to thank my advisors, Dr. Kang James and Dr. Barry James, for their guidance and support throughout this entire project. Contents 1. Introduction 1 2. Nearest Neighbor Estimators 3 2.1 Nearest-Neighbor Information 3 2.2 Nearest-Neighbor Regression Estimator Overview 4 2.4 Nearest-Neighbor Regression Estimator Derivation 5 2.5 Nearest-Neighbor Maximum Likelihood Estimator 10 2.6 Revised Nearest-Neighbor Maximum Likelihood Estimator 13 3. Datasets 14 3.1 Gaussian Sphere 14 3.2 Swiss Roll 14 3.3 Double Swiss Roll 15 3.4 Artificial Face 16 3.5 25-Dimensional Gaussian Sphere (ID = 15) 16 4. Results 17 4.1 Accuracy 17 4.2 Dependence on Number of Neighbors 19 4.3 Dependence on Distribution Type 20 4.4 Effectiveness on Datasets with High Intrinsic Dimension 21 4.5 Summary 22 References 23 A. Appendix 24 A.1 Program Overview 24 A.2 Code 25 int_dim.cpp 25 int_dim_reg