一种基于Hadoop的个性化推荐系统架构.PDF

2012年第2卷 第8期 7 一种基于 Hadoop 的个性化推荐系统架构# 张永霞，王洪波，程时端** （北京邮电大学网络与交换技术国家重点实验室，北京 100876） 摘要：随着计算机技术的发展和互联网的快速普及，如何快速的从海量数据中获取用户想要 的信息逐步成为用户关注的焦点之一。个性化推荐系统应运而生，通过获取用户在互联网上 的日志信息，分析用户的喜爱偏好，从而为用户推荐其可能感兴趣的信息。然而，随着互联 网的发展，互联网上充斥的用户日志信息越来越多，个性化推荐系统面临着存储空间的可扩 展性与分析计算的效率等瓶颈问题，单纯依靠提升计算机存储空间和计算性能显然不能从根 本上解决问题。本文针对该问题，以基于 Hadoop 分布式文件存储 HDFS 和分布式计算框架 Map/Reduce 的工具为基础，针对本文的应用场景对Mahout 实现的推荐算法做了优化，实现 了一种基于分布式计算框架Hadoop 之上的个性化推荐系统架构，并通过实验证明该方案解 决了数据存储的可扩展性和计算性能方面的问题，一定程度上提高了推荐的准确率。 关键词：计算机应用技术；分布式计算；海量数据；个性化推荐；Hadoop 中图分类号：TP393.1 A personalized recommendation system framework based on Hadoop Zhang Yongxia, Wang Hongbo, Cheng Shiduan (State Key Laboratory of Networking and Switching Technology, Beijing University of Posts Telecommunications, Beijing 100876) Abstract: As the increase of the network data, how to obtain the information quickly is becoming one of the most important problem for users. Personalized recommendation system is used to analysis the users behavior log and get the users favorite preferences, then help the users get what they want fastly. However, more and more information makes to produce a series of problems, such as the scalability,the computing performance. To solve this problem, this paper proposes a solution that designing and implementing a personalized recommendation system framework based on Hadoop, and optimizing the code of the Mahout to improve the accuracy rate of the recommendation system. The experiments results validate that the recommendation accurac