2016-Adversarial Training Methods for Semi-Supervised Text Classification英文文献资料.pdf

Under review as a conference paper at ICLR 2017 ADVERSARIAL TRAINING METHODS FOR SEMI -SUPERVISED TEXT CLASSIFICATION 1,2 2 3 Takeru Miyato , Andrew M Dai , Ian Goodfellow takeru.miyato@, adai@, ian@ 1 Kyoto University, 2 Google Brain and 3 OpenAI 6 1 ABSTRACT 0 2 Adversarial training provides a means of regularizing supervised learning algo- v rithms while virtual adversarial training is able to extend supervised learning al- o gorithms to the semi-supervised setting. However, both methods require making N small perturbations to numerous entries of the input vector, which is inappropri- 7 ate for sparse high-dimensional inputs such as one-hot word representations. We extend adversarial and virtual adversarial training to the text domain by applying ] perturbations to the word embeddings in a recurrent neural network rather than L to the original input itself. The proposed method achieves state of the art results M on multiple benchmark semi-supervised and purely supervised tasks. We provide . visualizations and analysis showing that the learned word embeddings have im- t proved in quality and that while training, the model is less prone to overfitting. a t s [ 1 INTRODUCTION 2 v Adversarial examples are examples that are created by making small perturbations to the input de- 5