牛津大学机器学习12.pdfVIP

Reinforcement learning Nando de Freitas The Promise of Reinforcement Learning Learning to act through trial and error. Environment { observation, reward } { action } • An agent interacts with an Agent environment and learns by maximizing a scalar reward signal. • No models, labels, demonstrations, or any other human-provided supervision signal. • Representation has been a challenge/missing. [Volodymyr Mnih] Deep Reinforcement Learning • Combining deep neural networks with RL. • Learn to act from high-dimensional sensory inputs. • Is a noisy, sparse, and delayed reward signal sufficient for training deep networks? Credit assignment problem. Environment { observations, reward } { action } [Volodymyr Mnih] Example: Learning to play Atari David Silver Direct policy search example: Attention [Ba, Mnih, Kavuckuoglu] Results MNIST Street View House sequences Number sequences • The attention-based model achieves state-of-the-art accuracy on the SVHN multi-digit task - 3.9% error. • 4 times fewer floating point operations than the best ConvNet. [Volodymyr Mnih et al] Attention-Based Game Agent • Roughly the same model and training method can be used in a game-playing agent. • The agent learns to track a ball without being told to do so.