Evaluation of Two Connectionist Approaches to Stack Representation.pdfVIP

Evaluation of Two Connectionist Approaches to Stack Representation Reb ecca Hwa TR Center for Research in Computing Technology Harvard University Cambridge Massachusetts Evaluation of Two Connectionist Approaches to Stack Representation Reb ecca Hwa Abstract This study empirically compares two distributed connectionist learn ing mo dels trained to represent an arbitrarily deep stack One is Pol lacks Recursive AutoAsso ciative Memory a recurrent back propa gating neural network that uses a hidden intermediate representation The other is the Exp onential Decay Mo del a novel architecture that we prop ose here which tries to learn an explicit represention that mo dels the stack as an exp onentially decaying entity We show that although the concept of a stack is learnable for b oth approaches neither mo del is able to deliver the arbitrary depth attribute Ultimately b oth suf fer from the rapid rate of error propagation inherent in their recursive structures Intro duction A stack is one of the most fundamental abstract datastructures in Computer Science it forms the b edro ck of traditional symb olic systems And yet when we move into a distributed connectionist framework nding a go o d representation for such a basic datastructure is surprisingly dicult One could of course hand co de a neural network to mimic the b ehavior of a stack under the symb olic system eg some form of a xed sized shift register but that would circumvent the advantages of connectionism such as the abilit