基于带注意力机制CNN的联合知识表示模型Knowledge-计算语言学.PDF

文章编号：1003-0077 （2011）00-0000-00 基于带注意力机制CNN 的联合知识表示模型 彭敏，姚亚兰，谢倩倩，高望 （武汉大学计算机学院，武汉 430072 ） 摘要：知识表示学习在自然语言处理领域获得了广泛关注，尤其在实体链指、关系抽取及自动问答等任务 上表现优异。然而，大部分已有的表示学习模型仅利用知识库中的结构信息，无法很好地处理新的实体或 关联事实极少的实体。为解决该问题，本文提出了引入实体描述信息的联合知识表示模型。该模型先利用 卷积神经网络编码实体描述，然后利用注意力机制来选择文本中的有效信息，接着又引入位置向量作为补 充信息，最后利用门机制联合结构和文本的向量，形成最终的联合表示。实验表明，本文的模型在链路预 测和三元组分类任务上与目前最好的模型性能相近。 关键词：知识表示学习；卷积神经网络；注意力机制 中图分类号：TP391 文献标识码：A Knowledge representation learning for joint structural and textual embed-ding via Attention-based CNN Min Peng, Yalan Yao, Qianqian Xie, Wang Gao (School of Computer, Wuhan University, Wuhan 430072, China) Abstract : Knowledge representation learning has attracted lots of attention in natural language processing with especially encouraging results on tasks such as Entity Linking, Relationship Extraction, Question Answering and so on. However, most of the existing models only use the structural information of knowledge graph and cannot handle new entities or entities with few facts well. Hence, we propose a joint knowledge representation model which utilizes both entity description and structural information. Firstly, we introduce convolutional neural network models to encode the entity description. Then, we design the attention mechanism to select the valid information of the text. Moreover, we introduce the position vector as the supplementary information. Finally, a gating mechanism is used to integrate the structural and textual information into the joint representation. Experimental results show that our models outperform other baselines on link prediction and triplet classification tasks. Key words: knowledge representation learning; CNN; attention mechanism 1 引言 目前，知识库在智能问答及个性推荐等人工智能领域应用前景广泛。知识库通常表示成 网络结构，使用三元组（头实体，关系，