语言建模中最小化样本风险算法的研究和改进-软件学报.pdf

ISSN 1000-9825, CODEN RUXUEW E-mail: jos@ Journal of Software, Vol.18, No.2, February 2007, pp .196−204 http://www.j DOI: 10. 1360/jos1 80196 Tel/Fax : +86-10 © 2007 by Journal of Software. All rights reserved. ∗ 语言建模中最小化样本风险算法的研究和改进 1+ 2 1 袁 伟 , 高剑峰 , 步丰林 1(上海交通大学 计算机科学与工程系,上海 200230) 2(Natural Language Processing Group, Microsoft Research, Redmond 98052,USA) A Study and Improvement of Minimum Sample Risk Methods for Language Modeling 1+ 2 1 YUAN Wei , GAO Jian-Feng , BU Feng-Lin 1(Department of Computer Science and Engineering, Shanghai Jiaotong University, Shanghai 200230, China) 2(Natural Language Processing Group, Microsoft Research, Redmond 98052, USA) + Corresponding author: Phn: +86-21 E-mail: weiyuan1981@ Yuan W, Gao JF, Bu FL. A study and improvement of minimum sample risk methods for language modeling. Journal of Software, 2007,18(2):196−204. /1000-9825/18/196.htm Abstract: Most existing discriminative training methods adopt smooth loss functions that could be optimized directly. In natural language processing (NLP), however, many applications adopt evaluation metrics taking a form as a step function, such as character error rate (CER). To address the problem, a newly-proposed discriminative training method is analyzed, which is called minimum sample risk (MSR). Unlike other discriminative methods, MSR directly takes a step function as its loss function. MSR is firstly analyzed and improved in time/space complexity. Then an improved