基于WEB页面的关键词与关键概念提取技术-计算机软件与理论专业论文.docxVIP

AbstractAbst Abstract Abst ract Keyword Extraction is an imponant technique of text infOrrnation process．At present，Keyword Extraction is an important technique used for automatic abstract， automatlc classifjcation，subject extraction，subject word extraction etc． The paper introduces a new technique of keyword extrac石on and key concept cxtraction based on Web page，the design and implement of expe—mental system，and me application of the system in the 3earch en百ne The paper includes three main pan． First，Keyword Extraction System．The paper desc^bes the special of Web page compared with the common text．Depending on the special，a technique of keyword cxtraction based on W色b page is introduced The system takes fUll adVantage of tags Jn theW曲page． Second，l(ey Concept Extraction System．Language is a deVeloping culture，and T1ew concepts are pmduced．And mally proper names which incIude person name， geogmphy name and corporation name，are new unknown concept．These concepts have an impact on the result of K|eyword Extraction system．The paperbrings forWard a key concept extraction technique based on the mutual info舯ation and context dependency．The means avoids the锄mcation eff-ect of N—gram model and realizes vari—gram statistical model of concept extraction．At the safne tlme，the paper adopts the way based on mles to optiIIlize the extmction result． In the end，a simple research is done for the application of the system in the Sear【：h engine．By analyzing the relevance of search engine，the paper brings fonVard a impmVed system relevance model and desc—bes the design of the model． Key-ord keyword；key cOncept；SeaI℃h engine ．Ⅱ． 第1章绪论第1章绪论 第1章绪论 第1章绪论 1．1课题性质 在“中文智能搜索引擎”(国家863项目)己日益趋于稳定和完善的同时， 我们课题组又进一步提出了提高搜索速度、优化搜索结果的新要求，提高搜索结 果的相关度及查询速度，尽量减少无关结果或低相关结果对用户造成的不便。 1．2课题研究的背景 随着信息科技的进步和互联网的日益普及，人类正在进行信息史上最巨大的 一项工程，即将现实世界现有的信息，诸如报纸、期刊、书籍、专利文献等，都 放到网络上去，同时也不停地在网络上生产出数不胜数的新信息。任何人在任何 时间、任何地点都可以通过网络发布任何信息。整个网络正在堆积成一个前所未 有的超级大型数据库，也就是说网络已经成为一个庞大而杂乱无章的桌面图书 馆。面对潮水般涌来的电子文献，人们迫切需要能够自动实现信息采集、过滤、 整理