Probabilistic Robotics Stanford University概率机器人斯坦福大学.pptVIP

Probabilistic Robotics Problem Classes Deterministic vs. stochastic actions Full vs. partial observability Deterministic, fully observable Stochastic, Fully Observable Stochastic, Partially Observable Markov Decision Process (MDP) Markov Decision Process (MDP) Rewards and Policies Policy (general case): Policy (fully observable case): Expected cumulative payoff: T=1: greedy policy T1: finite horizon case, typically no discount T=infty: infinite-horizon case, finite reward if discount 1 Policies contd. Expected cumulative payoff of policy: Optimal policy: 1-step optimal policy: Value function of 1-step optimal policy: 2-step Policies Optimal policy: Value function: T-step Policies Optimal policy: Value function: Infinite Horizon Optimal policy: Bellman equation Fix point is optimal policy Necessary and sufficient condition Value Iteration for all x do endfor repeat until convergence for all x do endfor endrepeat Value Iteration for Motion Planning Value Function and Policy Iteration Often the optimal policy has been reached long before the value function has converged. Policy iteration calculates a new policy based on the current value function and then calculates a new value function based on this policy. This process often converges faster to the optimal policy. * * SA-1 Planning and Control: Markov Decision Processes s2 s3 s4 s5 s1 0.7 0.3 0.9 0.1 0.3 0.3 0.4 0.99 0.01 0.2 0.8 r=-10 r=20 r=0 r=1 r=0 Given: States x Actions u Transition probabilities p(x‘|u,x) Reward / payoff function r(x,u) Wanted: Policy p(x) that maximizes the future expected reward