LearningaDeepHybridModelforSemi-SupervisedText.PDFVIP

Learning a Deep Hybrid Model for Semi-Supervised Text Classi?cation Alexander G. Ororbia II, C. Lee Giles, David Reitter College of Information Sciences and Technology The Pennsylvania State University, University Park, PA {ago109, giles, reitter}@ Abstract and supervised learning frameworks are no longer applicable. While incremental approaches such We present a novel ?ne-tuning algorithm as co-training have been employed to help these in a deep hybrid architecture for semi- models learn in a more update-able fashion (Blum supervised text classi?cation. During and Mitchell, 1998; Gollapalli et al., 2013), neural each increment of the online learning pro- architectures can naturally be trained in an online cess, the ?ne-tuning algorithm serves as manner through the use of stochastic gradient de- a top-down mechanism for pseudo-jointly scent (SGD). modifying model parameters following a bottom-up generative learning pass. The Semi-supervised online learning does not only resulting model, trained under what we address practical applications, but it also re?ects call the Bottom-Up-Top-Down learning al- some challenges of human category acquisition gorithm, is shown to outperform a vari- (Tomasello, 2001). Consider the case of a child ety of competitive models and baselines learning to discriminate between object categories trained across a wide range of splits be- and mapping them to words, given only a small tween supervised and unsupervised train- amount of explicitly labeled data (the mother ing data. pointing to the object), and a large portion of un- supervised learning, where the child comprehends 1 Introduction an adult’s speech or experi