强化学习基本概念与训练算法.docVIP

Reinforcement Learning --basic conceptions By ZJC 2018.05 Learning Methods ? Supervised Learning：explicit signals ? Unsupervised Learning：no signals ? Semi-Supervised Learning：partial/weak signals Learning Methods ? Supervised Learning：explicit signals 1. The training data is instance, label. 2. The general objective is minimizing the gap between “prediction” and “label”. 3. Most of the labels are from manual work. Learning Methods ? Unsupervised Learning：no signals 1. The training data is instance. 2. The general objective is finding some potential patterns under some heuristic criterions. Learning Methods ? Semi-Supervised Learning：partial/weak signals 1. Partial data has labels 2. The signals are delayed/weak. Reinforcement Learning ? Reinforcement Learning(RL) is a kind of semi- supervised approaches 1. RL is a framework 2. RL is an automation perspective of AI Pac-Man 1. A sequential decision problem. 2. Delayed rewards and the long-term goal. Basic Conceptions of RL Agent: 1. take an action at each step conditioned on the current state. 2. After the action, agent gets the reward, and the state changes. Figure borrowed from Yu Yang(NJU) Basic Conceptions of RL Environment: 1. provide the reward signals to the agent 2. is part of the state Figure borrowed from Yu Yang(NJU) Basic Conceptions of RL Goal of the agent: take right actions(policy) to achieve the maximum long-term total rewards(return). Figure borrowed from Yu Yang(NJU) Formulation of RL 1. States 2. Actions 3. Rewards 4. Policy Figure borrowed from Yu Yang(NJU) Agent Decision Process r1 s s a a1 r2 … s r3 s2 t a2 at rt+1 rt+2 … s a Reward Return Return：all the rewards. Finite steps return： Infinite steps return： is the discount factor. Average return： Policy Function Policy：The agent ’s decision strategy (stable and stochastic). Deterministic Policy： Stochastic Policy： Search Space： Utility/Value Function Value：The expected return that the agent starts from a certain state s. state-value functio