第10章_强化学习.ppt

高级人工智能第十章 ;内容提要;引言;引言;引言;引言;内容提要;主体;描述一个环境（问题）;强化学习问题;与监督学习对比;强化学习要素;在策略Π下的Bellman公式;Bellman最优策略公式;马尔可夫决策过程 MARKOV DECISION PROCESS;马尔可夫决策过程 MARKOV DECISION PROCESS;MARKOV DECISION PROCESS;马尔可夫决策过程 MARKOV DECISION PROCESS;;MDP Graphical Representation;Reinforcement Learning …;Reinforcement Learning …;动态规划Dynamic Programming;动态规划Dynamic Programming - Problem;Finite Horizon Problem Infinite Horizon Problem Value Iteration ;动态规划中的策略迭代/值迭代;动态规划方法;自适应动态规划(ADP);Value Iteration Algorithm;Policy Iteration Algorithm ;动态规划Dynamic Programming;没有模型的方法Model Free Methods;蒙特卡罗方法 Monte Carlo Methods ;Monte Carlo方法;蒙特卡罗方法 Monte Carlo Methods ;Monte Carlo策略评价; Problem: Unvisited s, a pairs (problem of maintaining exploration) For every s, a make sure that: P(s, a selected as a start state and action) 0 (Assumption of exploring starts ) ;蒙特卡罗控制;时序差分学习 Temporal-Difference;时序差分学习 TD;时序差分学习 Temporal-Difference;时序差分学习 (TD);TD(l) – A Forward View;时序差分学习算法 TD(?);时序差分学习算法 TD(?);时序差分学习算法;时序差分学习算法收敛性TD(?);时序差分学习 TD;Q-learning;Q-learning;Q-Learning;Q-learning;Exploration;Common exploration methods;Q-Learning Algorithm;Q-Learning Algorithm;What is the intuition?;A-Learning ;A-Learning Algorithm (Inefficient Version);Differences between Q and A-learning;Q-Learning Vs. A-Learning;POMDP部分感知马氏决策过程 ;Framework of POMDP;POMDPs;POMDPs – A Solution Strategy;POMDP_信度状态方法;The Learning Process of Belief MDP;Major Methods to Solve POMDP ;强化学习中的函数估计;并行两个迭代过程;Function Approximator: V( s) = f( s, w) Update: Gradient-descent Sarsa: w ? w + a [rt+1 + g Q(st+1,at+1)- Q(st,at)] ?w f(st,at,w);Semi-MDP;The equations;Multi-agent MDP;三种观点;马尔可夫对策;基于平衡解方法的强化学习;Applications of RL;仿真机器人足球;仿真机器人足球;仿真机器人足球;仿真机器人足球;仿真机器人足球;仿真机器人足球;仿???机器人足球;强化学习应用_调度;应用 - 分布式电梯控制;Q(“Publications”, “My research”, “Will’s FrontDoor Page”);强化学习应用_自适应控制生物信息学