2004 informs continuous time discounted jump markov decision processes a discrete-event ap文档.pdf

MATHEMATICS OF OPERATIONS RESEARCH informs Vol. 29, No. 3, August 2004, pp. 492–524 ® issn 0364-765X eissn 1526-5471 04 2903 0492 doi 10.1287/moor.1040.0089 © 2004 INFORMS Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach Eugene A. Feinberg Department of Applied Mathematics and Statistics, SUNY at Stony Brook, Stony Brook, New York 11794-3600, eugene.feinberg@sunysb.edu This paper introduces and develops a new approach to the theory of continuous time jump Markov decision processes (CTJMDP). This approach reduces discounted CTJMDPs to discounted semi-Markov decision processes (SMDPs) and eventually to discrete-time Markov decision processes (MDPs). The reduction is based on the equivalence of strategies that change actions between jumps and the randomized strategies that change actions only at jump epochs. This holds both for one-criterion problems and for multiple-objective problems with constraints. In particular, this paper introduces the theory for multiple-objective problems with expected total discounted rewards and constraints. If a problem is feasible, there exist three types of optimal policies: (i) nonrandomized switching stationary policies, (ii) randomized stationary policies for the CTJMDP, and (iii) randomized stationary policies for the corresponding SMDP with exponentially distributed sojourn times, and these policies can be implemented as randomized strategies in the CTJMDP. Key words : continuous time jump Markov decision process; semi-Markov decision process MSC2