2010 State Aggregation based Linear Programming approach to Approximate Dynamic Programming外文翻译.pdfVIP

49th IEEE Conference on Decision and Control December 15-17, 2010 Hilton Atlanta Hotel, Atlanta, GA, USA State Aggregation based Linear Programming approach to Approximate Dynamic Programming S. Darbha, K. Krishnamoorthy, M. Pachter and P. Chandler Abstract— One often encounters the curse of dimensionality It is well known that sometimes the linear program used in the application of dynamic programming to determine in these techniques may be overly constrained leading to optimal policies for controlled Markov chains. In this paper, poor approximations and perhaps even infeasibility [8]. An we provide a method to construct sub-optimal policies along with a bound for the deviation of such a policy from the efficient and therefore attractive basis function approach with optimum through the use of restricted linear programming. guaranteed feasibility and bounds on the approximation error The novelty of this approach lies in circumventing the need is proposed in [9]. Unfortunately it is not clear how one for a value iteration or a linear program defined on the should go about choosing the basis functions for a generic entire state-space. Instead, the state-space is partitioned based problem. Within the basis function framework, we propose on the reward structure and the optimal cost-to-go or value function is approximated by a constant over each partition. We to partition the state space and compute approximate value associate a meta-state with each partition, where the transition functions associated with each partition. In particular, we probabilities between these meta-states can be derived from use a state aggregation scheme