factored conditional restricted boltzmann machines for.pdfVIP

Factored Conditional Restricted Boltzmann Machines for Modeling Motion Style Graham W. Taylor GWTAYLOR @CS .TORONTO .EDU Geoffrey E. Hinton HINTON @CS .TORONTO .EDU Department of Computer Science, University of Toronto, Toronto, Ontario M5S 2G4, Canada Abstract proximate learning algorithm called contrastive divergence (CD) (Hinton, 2002). RBMs have been used in a variety The Conditional Restricted Boltzmann Machine of applications (Hinton Salakhutdinov, 2006; Salakhut- (CRBM) is a recently proposed model for time dinov et al., 2007) and their properties have become better series that has a rich, distributed hidden state understood over the last few years (Welling et al., 2005; and permits simple, exact inference. We present Carreira-Perpinan Hinton, 2005; Salakhutdinov Mur- a new model, based on the CRBM that pre- ray, 2008). The CD learning procedure has also been im- serves its most important computational proper- proved (Tieleman, 2008). ties and includes multiplicative three-way inter- actions that allow the effective interaction weight A major motivation for the use of RBMs is that they can be