complex valued recurrent neural network from architecture to training复杂的价值从架构到递归神经网络的训练.pdfVIP

Journal of Signal and Information Processing, 2012, 3, 192-197 /10.4236/jsip.2012.32026 Published Online May 2012 (http://www.SciRP.org/journal/jsip) Complex Valued Recurrent Neural Network: From Architecture to Training 1,2 1 2 Alexey Minin , Alois Knoll , Hans-Georg Zimmermann 1Technische Universität München—Robotics and Embedded Systems, München, Germany; 2Siemens AG, Corporate Technology, München, Germany. Email: alexey.minin@ Received December 7th, 2011; revised February 11th, 2012; accepted March 18th, 2012 ABSTRACT Recurrent Neural Networks were invented a long time ago, and dozens of different architectures have been published. In this paper we generalize recurrent architectures to a state space model, and we also generalize the numbers the net- work can process to the complex domain. We show how to train the recurrent network in the complex valued case, and we present the theorems and procedures to make the training stable. We also show that the complex valued recurrent neural network is a generalization of the real valued counterpart and that it has specific advantages over the latter. We conclude the paper with a discussion of possible applications and scenarios for using these networks. Keywords: Complex Valued Neural Networks; Complex Valued System Identification; Recurrent Neural Networks; Complex Valued Recurrent Neural Networks 1. Introduction dynamical systems. There are two principle ways of modeling dynamical systems: 1) use a feed-forward neu- Current paper aims to give the complete guidance from ral network and use delayed inputs or 2) use a recurrent the state space models