Using Learned Policies in HeuristicSearch Planning.pdfVIP

Using Learned Policies in Heuristic-Search Planning SungWook Yoon Alan Fern Robert Givan Computer Science Engineering Computer Science Department Electrical Computer Engineering Arizona State University Oregon State University Purdue University Tempe, AZ 85281 Corvallis, OR 97331 West Lafayette, IN 47907 Sungwook.Yoon@asu.edu afern@cs.orst.edu givan@purdue.edu Abstract limit the size of the training data, as occurs in planning competitions. Nevertheless such policies still capture use- Many current state-of-the-art planners rely on forward heuris- ful, though imperfect, constraints on good courses of action. tic search. The success of such search typically depends The main goal of this paper is to develop and evaluate an ap- on heuristic distance-to-the-goal estimates derived from the plangraph. Such estimates are effective in guiding search for proach for combining such imperfect policies and heuristics many domains, but there remain many other domains where in order to improve over the performance of either alone. current heuristics are inadequate to guide forward search ef- There are techniques that can improve on imperfect poli- fectively. In some of these domains, it is possible to learn cies. Policy rollout (Bertsekas Tsitsiklis 1996) an