A Convolutional Neural Network for Modelling Sentences.pdf

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette {nal.kalchbrenner, edward.grefenstette, phil.blunsom}@cs.ox.ac.uk Department of Computer Science University of Oxford Phil Blunsom Abstract The ability to accurately represent sen- tences is central to language understand- ing. We describe a convolutional architec- ture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pool- ing, a global pooling operation over lin- ear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily ap- plicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment predic- tion, six-way question classification and Twitter sentiment prediction by distant su- pervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline. 1 Introduction The aim of a sentence model is to analyse and represent the semantic content of a sentence for purposes of classification or generation. The sen- tence modelling problem is at the core of many tasks involving a degree of natural language com- prehension. These tasks include sentiment analy- sis, paraphrase detection, entailment recognition, summarisation, discourse analysis, machine trans- lation, grounded language learning and image re- trieval. Since individual sentences are rarely ob- served or not observed at all, one must represent a sentence in terms of features that depend on the words and short n-grams in the sentence that are frequently observed. The core of a sentence model involves a feature function that defines the process The cat sat on the red mat The cat sat on the red