基于词向量特征的循环神经网络语言模型.pdfVIP

第28 卷摇 第4 期摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 模式识别与人工智能摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 Vol.28摇 No.4 摇 2015年4月摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 PR AI摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 Apr.摇 2015 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 摇 基基于词向量特征的循环神经网络语言模型* 张摇 剑摇 摇 屈摇 丹摇 摇 李摇 真 (中国人民解放军信息工程大学 信息系统工程学院摇 郑州450001) 摘摇 要摇 循环神经网络语言模型能解决传统N鄄gram模型中存在的数据稀疏和维数灾难问题,但仍缺乏对长距离 信息的描述能力.为此文中提出一种基于词向量特征的循环神经网络语言模型改进方法.该方法在输入层中增加 特征层,改进模型结构.在模型训练时,通过特征层加入上下文词向量,增强网络对长距离信息约束的学习能力.实 验表明,文中方法能有效提高语言模型的性能. 关键词摇 语音识别,语言模型,循环神经网络,词向量 中图法分类号摇 TP391 DOI摇 10.16451/ ki.issn1003鄄6059.201504002 Recurrent Neural Network Language Model Based on Word Vector Features ZHANGJian,QU Dan,LI Zhen (Institute of Information System Engineering,The PLA Information Engineering University, Zhengzhou450001) ABSTRACT The recurrent neural network language model(RNNLM) solves the problems of data sparseness and dimensionality disaster in traditional N鄄gram models. However,the original RNNLM is still lack of long dependence due to the vanishing gradient problem. In this paper, an improved method based on contextual word vectors is proposed for RNNLM. To improve the structure of models,a feature layer is added into the input layer. Contextual word vectors are added into the model with feature layer to reinforce the ability of learning long-distance information during the training. Experimental results show that the proposed method effectively improves the performance of RNNLM. Key Words摇 Speech Recognition,Language Model,Recurrent Neural Network,Word Vector *国家863计划项目(No.2012AA011603)、国家自然科学基金项目(No资助 收稿日期:2014-02-27;修回日期: