evaluating collaborative filtering recommender systems（评估协同过滤推荐系统）.pdfVIP

© ACM, 2004. This is the authors version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Information Systems (TOIS), {vol. 22, no. 1, (January 2004} /10.1145/963770.963772 Evaluating Collaborative Filtering Recommender Systems JONATHAN L. HERLOCKER School of Electrical Engineering Computer Science, Oregon State University and JOSEPH A. KONSTAN, LOREN G. TERVEEN, AND JOHN T. RIEDL GroupLens Research Group, University of Minnesota ________________________________________________________________________ Recommender systems have been evaluated in many, often incomparable, ways. In this paper we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated. Categories and Subject Descriptors: H.3.4 [Information Storage and Retrieval]: Systems and Software – Performance Evaluation (efficiency and effectiveness) General Terms: Experimentation, Measurement, Performance Additional Key Words and Phrases: Collaborative filtering, recommender systems, metrics, evaluation ________________________________________________________________________ 1. INT