linear programming solvers for markov decision strongprocessesstrong.pdfVIP

Linear Programming solvers for Markov Decision Processes ´ ˜ Diego Bello and German Riano* Abstract— This paper describes linear programming solvers The remainder of this paper is organized as follows. In for Markov Decision Processes, as an extension to the JMDP Section II we give a brief description of Markov Decision program. JMDP is an object-oriented framework to model Processes and the linear programs that can be used to solve and solve Markov Decision Processes (MDP) programmed in Java. The developed solvers work for the Discounted Cost and them. In Section III we give a brief description of JMDP. Average Cost criteria. Our solvers are compared with existing In Section IV we explain how the solvers proposed were Value Iteration and Policy Iteration solvers. implemented in an object-oriented language programmed in Java. Section V shows the computational performance of I. INTRODUCTION the implemented linear programming solvers and in the last This paper describes linear programming solvers for section VI we give concluding remarks. Markov Decision Processes, as an extension to the JMDP program [1]. JMDP is an object-oriented framework to model II. MARKOV DECISION PROCESS and solve Markov Decision Processes (MDP) programmed The present framework is based on [8], [2], [3] and [9]. in Java. MDPs are powerful modeling tools that allow Consider a