Web语料库建设初探.pdfVIP

第27卷　第5期 计　算　机　工　程 2001年5月 Vol.27 № 5 Computer Engineering May 2001 ·开发研究与设计技术· 文章编号：1000—3428(2001)05—0177—02 文献标识码：A 中图分类号： TP317.2 Web语料库建设初探 俞倩兰，温晓行 (江苏常熟高等专科学校机电工程系，常熟215500) 摘 要：对网上中文信息语料库搜集技术的实现原理和关键技术进行了讨论和分析，介绍了基于Web 网络的通信及网上自动获取信息的原 理，讨论了中文信息处理中的分词技术及其发展，提出了一个网上《人民日报》语料库搜集技术的实现方案。 关键词：Web ；语料库；分词 A Preliminary Study on the Construction of Web Corpus YU Qianlan，WEN Xiaoxing (Department of Mechanics and Electronics,Changshu College,Changshu 215500) Abstract With the Internet getting increasingly popular and Chinese information on WWW becoming ever greater in volume, the 【 】 importance of automatic data search technique in Chinese information corpus on the line is more obvious than ever. The development and improvement of the technique is of great significance for bettering the process level of Chinese information.This paper, based on a discussion and analysis of the realization laws and essential technology of data search technique in the Chinese information corpus, attempts to introduce the principles of realizing net communication at the Web and obtaining automatically the information on the line. A scheme of search technique for the corpus of Peoples Daily is suggested with the classification and combination technology so far developed in the process of Chinese information discussed and analyzed. Key words Web Corpus Dividing words 【 】 ； ； 1 概述 的静态特性，无法反映整个语料收集过程中的动态变化，因 语料又称素材，是自然发生的语言材料（书面语和口 此同理想的水平存在较大差