reinforcement learning on slow features of high-dimensional input streams强化学习在低速高维输入流的特性.pdfVIP

Reinforcement Learning on Slow Features of High-Dimensional Input Streams 1 2,3 2,3,4 Robert Legenstein *, Niko Wilbert , Laurenz Wiskott ¨ 1 Institute for Theoretical Computer Science, Graz University of Technology, Graz, Austria, 2 Institute for Theoretical Biology, Humboldt-Universitat zu Berlin, Berlin, ¨ ¨ Germany, 3 Bernstein Center for Computational Neuroscience, Berlin, Germany, 4 Institut fur Neuroinformatik, Ruhr-Universitat Bochum, Bochum, Germany Abstract Humans and animals are able to learn complex behaviors based on a massive stream of sensory information from different modalities. Early animal studies have identified learning mechanisms that are based on reward and punishment such that animals tend to avoid actions that lead to punishment whereas rewarded actions are reinforced. However, most algorithms for reward-based learning are only applicable if the dimensionality of the state-space is sufficiently small or its structure is sufficiently simple. Therefore, the question arises how the problem of learning on high-dimensional data is solved in the brain. In this article, we propose a biologically plausible generic two-stage learning system that can directly be applied to raw high-dimensional input streams. The system is composed of a hierarchical slow feature analysis (SFA) network for preprocessing and a simple neural network on top that is trained based on rewards. We demonstrate by computer simulations that this generic architecture