基于汉语信息抽取模型的股市公告系统的设计与实现-上海交通大学.doc

汉语股市公告信息抽取系统的设计与实现 摘要 本文介绍了一个基于中文信息抽取模型的股市公告信息抽取系统（SBIES）的设计与实现。介绍了该系统的结构框架和分布图。讨论了汉语信息抽取模型的具体结构，构建了由自动分词、自动标注和模板填充三个阶段组成的简化模型。简单介绍了自动分词的常用算法和自动标注中的标注规范。重点探讨了模板填充的具体算法。文中分别讨论了采用基于规则的结构主义方法和基于语料库概率统计的功能主义方法。着重讨论了采用隐马尔科夫模型进行信息抽取的具体算法。对模型的参数获取算法作了讨论，改进了Baum-Welch算法以适应信息抽取的应用。对领域文本做了人工标注，通过计算机处理获取所需的统计数据。利用统计数据完善HMM模型。 关键词：信息抽取，隐马尔科夫模型，自然语言  THE DESIGN AND IMPLEMENTATION OF CHINESE STOCK BULLETIN INFORMATION EXTRACTION SYSTEM Abstract This article introduced the design and implementation of a Chinese IE Technology based stock bulletin information extraction system (SBIES). The framework and deployment of the system were described. The structure of the Chinese information extraction model was discussed in detail. We proposed a simplified 3 tiers IE model consisting of automatic word segmentation, automatic annotation, and template filling. The algorithms used in automatic word segmentation and annotation were briefly introduced while algorithms used in template filling were focused on. In this article, the rule-based structuralism methods and the corpus-based statistical functionalism methods were discussed respectively. The Hidden Markov Model (HMM) was introduced to extract information and the algorithm was explained at length. The algorithm for model parameter acquisition was also analyzed and the Baum-Welch iteration algorithm was modified. Domain texts were annotated manually to acquire statistical data via computation. With these data, HMM-based IE was implemented. KEY WORDS: information extraction, hidden Markov model, natural language目录 1 概述 1 1.1 信息抽取技术（Information Extraction） 1 1.1.1 信息抽取技术的发展 1 1.1.2 信息抽取技术的特点 3 1.2 股市公告信息抽取系统（SBIES） 3 1.2.1 应用背景 3 1.2.2 系统框架结构 4 1.2.3 系统分布结构 5 1.3 本文内容简介 5 2 信息抽取模块的设计 7 2.1 模块内部结构 7 2.1.1 串行化的模块内部结构及其问题 7 2.1.2 简化的高性能信息抽取模型 8 2.2 自动分词 9 2.3 词类自动标注 10 3 信息抽取的关键算法 13 3.1 传统语言学方法 13 3.1.1 基于规则的信息抽取算法 13 3.1.2 基于规则信息抽取面临的严峻问题 15 3.2 统计学方法 16 3.2.1 基于语料库的统计语言学方法 16 3.2.2 隐马尔科夫模型（HMM）简介 16 3.2.3 隐马尔科夫模型（HMM）在信息抽取上的应用 17 3.2.4 隐马尔