Convolutional Neural Networks for Sentence Classi.pdf

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751, October 25-29, 2014, Doha, Qatar. c?2014 Association for Computational Linguistics Convolutional Neural Networks for Sentence Classification Yoon Kim New York University yhk255@ Abstract We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vec- tors for sentence-level classification tasks. We show that a simple CNN with lit- tle hyperparameter tuning and static vec- tors achieves excellent results on multi- ple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the ar- chitecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification. 1 Introduction Deep learning models have achieved remarkable results in computer vision (Krizhevsky et al., 2012) and speech recognition (Graves et al., 2013) in recent years. Within natural language process- ing, much of the work with deep learning meth- ods has involved learning word vector representa- tions through neural language models (Bengio et al., 2003; Yih et al., 2011; Mikolov et al., 2013) and performing composition over the learned word vectors for classification (Collobert et al., 2011). Word vectors, wherein words are projected from a sparse, 1-of-V encoding (here V is the vocabulary size) onto a lower dimensional vector space via a hidden layer, are essentially feature extractors that encode semantic features of words in their dimen- sions. In such dense representations, semantically close words are likewise close—in euclidean or cosine distance—in the lower dimensional vector space. Convolutional neural networks (CNN) utilize layers with convolving filters that are applied to local features (LeCun et