Heuristic Search Value Iteration for POMDPs.pdfVIP

Heuristic Search Value Iteration for POMDPs Trey Smith and Reid Simmons Robotics Institute, Carnegie Mellon University {trey,reids}@ri.cmu.edu Abstract By employing all of these techniques, HSVI gains a num- ber of benefits. Its use of heuristic search forward from We present a novel POMDP planning algorithm an initial belief (aided by the new observation heuristic) called heuristic search value iteration (HSVI). avoids unreachable or otherwise irrelevant parts of the be- HSVI is an anytime algorithm that returns a lief space. Its representations for both bounds are compact policy and a provable bound on its regret with and well-suited to generalizing local updates: improving respect to the optimal policy. HSVI gets the bounds at a specific belief also improves them at neigh- its power by combining two well-known tech- boring beliefs. niques: attention-focusing search heuristics and Some weaknesses of HSVI are that it is relatively compli- piecewise linear convex representations of the cated, and its upper bound updates are a source of major value function. HSVI’s soundness and con- overhead that only becomes worthwhile on the larger prob- vergence have been proven. On some bench- lems we studied. mark problems from the literature, HSVI dis- plays speedups of greater than 100 with respect