基于分布式计算的新型协同过滤算法分析-analysis of new collaborative filtering algorithm based on distributed computing.docxVIP

摘要随着互联网技术的迅猛发展，各种信息铺天盖地的呈现在我们面前，在这些 海量信息中检索出自己感兴趣的信息越来越难，出现了所谓的“信息超载”现象。 个性化推荐技术是解决这个问题的有效途径，它可以通过分析用户的历史信息来 建立模型，为用户推荐其感兴趣的产品。在个性化推荐的研究领域中，协同过滤 算法是最成熟，也是应用最广泛的一种技术。伴随着推荐系统中用户数目和产品 数目的不断增加，现有的协同过滤算法在实践中暴露出了不少问题。本文主要针 对协同过滤算法展开了深入细致的分析和探讨，主要研究内容如下：经过分析研究后，发现基于项目的协同过滤算法，尤其是 Slope One 算法，不 仅易于实现，效率高，并且算法具有较高的推荐准确度，不足的是在处理用户间 关系的个性化推荐时表现不是很好。为了获得更好推荐效果，本文提出了一种新 型的基于项目和用户相似性的协同过滤算法。该算法的核心思想是将用户间的相 似性和 Slope O ne 算法巧妙的结合起来，首先，计算出用户间的平均相似性，然后， 将用户间平均相似性作为 Slope One 算法的权值计算预测评分。最后，根据预测所 得评分值为用户给出推荐。为了解决海量数据时计算复杂度的问题，本文还采用 了分布式计算的方式，在 Hadoop 平台下利用 MapReduce 编程模型实现了该新型 协同过滤算法。在标准的 MovieLens 数据集上对算法作了多次验证，实验结果表明本文提出 的基于项目和用户相似性的协同过滤算法可以有效提高个性化推荐系统的推荐准 确度，更适用于当今的社会型网络。关键字： 个性化推荐 协同过滤 Slope One 分布式计算AbstractWith the rapid development of internet technology, all kinds of information present in front of us. How to find out the information they are interested in the mass of information has become more and more difficult, the phenomenon of “information overload” arises. Personalized recommendation technology is an effective way to solve this problem, and it can recommend products for users by analyzing users history information. Collaborative filtering is the most widely used technology. However, with the gradual increase of the number of users and products, existing collaborative filtering algorithm has exposed a lot of problems. The main contents of this paper are as follows:After analysis, the Slope One algorithm is not only easy to achieve, but also efficient and effective. However, the Slope One algorithm performs not so well when dealing with personalized recommendation tasks which concern the relationship of users. In order to get a better recommendation results, this paper propose a new collaborative filtering algorithm based on items and the similarity between users. The core idea of the algorithm is a combination of the similarity between users and the Slope One algorithm. First, calculate the average similarity between users. Then, the Slope One algorithm predicts score