基于上下文的真词错误检查及其校对方法.pdfVIP

基于上下文的真词错误检查及校对方法* 陆玉清，洪宇，陆军，姚建民，朱巧明 苏州大学江苏省计算机信息处理重点实验室 {hongy,jyao }@ 摘 要：英文文本中的真词错误即输入的错词是和原词相似的另一个有效词。本文主要研究了对该类错误 的检测。通过从所要检测的单词的上下文中提取句法和语义两个方面的特征，运用文档频率和信息增益进 行特征筛选，实现了对上下文特征的有效提取。最终把判断该单词使用的正确与否看作分类问题，使用 Winnow 分类算法进行训练和测试。通过5 阶交叉验证，所收集的61 组混淆集的平均正确率与召回率分别 为96%，79.47%。 关键词：真词错误；特征筛选；混淆集；Winnow 算法 Context-dependent word error’s correcting and proofreading {hongy,jyao }@ Yu-Qing Lu, Yu Hong, Jun Lu, Jian-Min Yao, Qiao-Min Zhu Provincial Key Laboratory of Computer Information Processing Technology Soochow University, Suzhou, China, 215006 Abstract: Context-dependent word errors in English texts are errors in which one correctly spelled word is substituted for another. This paper mainly discussed how to detect this type of errors. In the work reported here, we represent a method that firstly we extract features of syntax and semantics from the word context, and then we use Document Frequency and Information Gain to do feature selection, realizing the effective extraction of contextual features. At last, we modeled the detecting task as a classification using the Winnow algorithm to train and test the sentences that are related with those confusion words. With five-across validation, the average precision and recall are respectively 96%, 79.47% according to the 61groups of confusion sets. Key words: context-dependent word error; feature selection; confusion sets; Winnow algorithm 1 引言 拼写错误是英文文本中较常见的错误。Kukich[1]将英文文本中的错误分为两种类型，一 类是上下文无关的错误(isolated-word error) ，也叫非词错误，即输入的词是一个无效的词， 在字典中不存在。这种错误不需借助上下文，只要和词典中的词进行匹配就能发现，例如 the 错拼为teh、reluctant 错拼为reluctent 。另一类是上下文有关的错误(context-dependent word error)，也叫真词错误，即输入的词是和原词相似