资讯类新闻套图系统.pdfVIP

2014 年 第23 卷 第 10 期 计 算 机 系 统 应 用 资讯类新闻套图系统① 1,2 1,2,3 江浩亮 , 左 春 1( 中国科学院软件研究所 软件工程技术研发中心, 北京 100190) 2( 中国科学院大学, 北京 100190) 3( 中科软科技股份有限公司, 北京 100190) 摘 要: 考虑到图片具有对事件诠释力强, 传播便利的特点, 研究了从大量数据密集的新闻Web 页面中自动提取 数据, 并组织成套图结构展现给用户. 基于页面模板实现动态页面抽取和解析, 处理转换为对应的套图数据结构. 基于余弦相关性对来自不同网站的新闻套图数据进行去重, 并根据相应的标准, 为数据集进行评分排序. 考虑巨 大的新闻数据和用户数量, 本系统基于hadoop 分布式平台, 满足系统的高可扩展性. 本文将详细描述我们的系统 设计和实现, 并公布在百度资讯图片栏目上的运行结果. 关键词: Web 信息提取; 动态数据集; 高可扩展性; 个性化推荐; 套图 Web Information Extraction and Knowledge Presentation System JIANG Hao-Liang1,2, ZUO Chun1,2,3 1(Software engineering center, Institute of Software, Chinese Academy Of Sciences, Beij ing 100190, China) 2(University of Chinese Academy of Sciences, Beij ing 100190, China) 3(Sinosoft, Beij ing 100190, China) Abstract: Considering the picture has the futures that a strong interpretation of events and convenient disseminating, this paper studies extraction of data from a large number of news web pages, and organizational structure chart presented to the users. It achieves dynamic pages based on page template extraction and analysis, processing converted to the corresponding sets of datastructure. Based on the news cosine correlation graph data sets from different sites are differentiated, and in accordance with the appropriate standards for data collection to score sorted. This system is based on hadoop distributed platform, considering the large number of users and imgsets. This paper will describe the design and implementation of our system in detail, and report the results of running the system on Baidu news image column. Key words: web information extraction; dynamic data set; highly scalable; personalized recommendations; imgset 网络上从不缺少新闻, 但是在过去以文字为主题