面向Spark图书借阅数据关联模型研究.PDFVIP

安徽农业大学学报, 2018, 45(4): 768-771 Journal of Anhui Agricultural University [DOI] 10.13610/ki.1672-352x002 网络出版时间：2018/8/27 17:05:10 [URL] /kcms/detail/34.1162.s1656.003.html 面向 Spark 的图书借阅数据关联模型的研究 1 2 3 高琪娟 ，刘 锴 ，陈 佳 (1. 安徽农业大学信息与计算机学院，合肥 230036 ；2. 安徽农业大学现代教育信息中心，合肥 230036 ； 3. 中国电信芜湖分公司高端聚类服务中心，芜湖 241003) 摘 要：为了方便读者能在海量的图书资源中快速有效的找到需要的书籍，利用MapReduce 框架分块处理，结 合关联分析 Apriori 算法，将数据挖掘技术应用到图书管理系统中。但需要多次扫描数据库和产生大量候选集，对 Hadoop 平台处理速度带来了巨大挑战，因此，针对传统的Apriori 算法，提出基于内存计算、弹性分布式数据集处理 的Spark 平台为读者推荐书籍，指引读者的借阅行为。 关键词：Apriori 关联规则；Spark 平台；图书借阅行为模式；频繁项集 中图分类号：TP391.3 文献标识码：A 文章编号：1672352X (2018)04076804 Research of associative model for libraries’ book lending data based on the spark 1 2 3 GAO Qijuan , LIU Kai , CHEN Jia (1. School of Information and Computer Science, Anhui Agricultural University, Hefei 230036; 2. Modern Educational Technology Center of Anhui Agricultural University, Hefei 230036; 3. High-standard clustering Service Center Department of China Telecom Co., Ltd., Wuhu 241003) Abstract: In order to search the required books from a tremendous amount of resources immediately for au- thors, we tried to use the method of MapReduce for dealing the process of block data, combining the algorithm of Apriori, and applying data mining technology to the library management system. But the method referring to above need scan database many times and emerge a large number of candidate set, which will produce tremendous chal- lenge to the speed with Hadoop processing method. Thus, compared to the tradition method of Apriori, there is a new method based on the memory and RDD to compute in Spark platfor