Dynamic nonuniform abstractions for approximate planning in large structured stochastic do.pdfVIP

Dynamic Nonuniform Ab stractions for Approximate Planning in Large Structure d Sto chastic Domains J Baum and AE Nicholson Scho ol of Computer Science and Software Engineer ing Monash Univers ity Clayton VICTORIA Australia phone f ax fjiribannngcs mona shedu au Abstract The theory of Markov Deci s ion Pro ce ss e s MDPs provide s algor ithms for generating an optimal p olicy For large domains the s e al gor ithms b ecome intractable and approximate solutions b ecome nece s sary In thi s pap er we extend previous work on approximate planning in large sto chastic domains by us ing automaticallygenerate d nonuniform abstractions which exploit the structure of the state space We cons ider a state space expre ss e d as a cross pro duct of s ets or dimens ions We obtain approximate solutions by varying the level of abstraction s elect ively ignor ing some of the dimens ions in some parts of the state space We de scr ib e a mo dication of a standard p olicy generation algor ithm for the now nonMarkovian deci s ion pro ce ss which recalculate s value s for nearby state s bas e d on a lo cally uniform abstraction for each state We pre s ent metho ds to automatically generate an initial abstraction bas e d on the domain structure and to automaticall y mo dify the nonuniform ab straction The change s to the abstraction are bas e d on b oth the current p olicy and the likeliho o d of encounter ing particular state s in the future thereby taking into account the agents changing circumstance s Intro duction For small