GENERALIZEVIARECURSION方案.PDFVIP

Published as a conference paper at ICLR 2017 MAKING NEURAL PROGRAMMING ARCHITECTURES GENERALIZE VIA RECURSION Jonathon Cai, Richard Shin, Dawn Song Department of Computer Science University of California, Berkeley Berkeley, CA 94720, USA jonathon,ricshin,dawnsong@ ABSTRACT Empirically, neural networks that attempt to learn programs from data have exhib- ited poor generalizability. Moreover, it has traditionally been difficult to reason about the behavior of these models beyond a certain level of input complexity. In order to address these issues, we propose augmenting neural architectures with a key abstraction: recursion. As an application, we implement recursion in the Neural Programmer-Interpreter framework on four tasks: grade-school addition, bubble sort, topological sort, and quicksort. We demonstrate superior generaliz- ability and interpretability with small amounts of training data. Recursion divides the problem into smaller pieces and drastically reduces the domain of each neu- ral network component, making it tractable to prove guarantees about the overall system’s behavior. Our experience suggests that in order for neural architectures to robustly learn program semantics, it is necessary to incorporate a concept like recursion. 1 INTRODUCTION Training neural networks to synthesize robust programs from a small number of examples is a chal- lenging task. The space of possible programs is extremely large, and composing a program that per- forms robustly on the infinite space of possible inputs is difficult—in part because it is impractical to obtain enough training examples to easily disambiguate amongst all possible programs. Never- theless, we would like the model to quickly learn