Proceedings of the 2001 Winter Simulation Conference.PDFVIP

Proceedings of the 2001 Winter Simulation Conference B. A. Peters, J. S. Smith, D. J. Medeiros, and M. W. Rohrer, eds. A MARKOV CHAIN PERSPECTIVE ON ADAPTIVE MONTE CARLO ALGORITHMS Paritosh Y. Desai Peter W. Glynn Department of Management Science and Engineering Stanford University Stanford, CA 94309, U.S.A. ABSTRACT This paper is organized as follows. In Section 2, we discuss an adaptive algorithm for computing expected This paper discusses some connections between adaptive reward to absorption in the finite state Markov chain setting. Monte Carlo algorithms and general state space Markov Section 3 describes a new adaptive algorithm for solving chains. Adaptive algorithms are iterative methods in which eigenvalue problems for non-negative matrices. Finally, previously generated samples are used to construct a more Section 4 discusses our two examples through the prism of efficient sampling distribution at the current iteration. In general state space Markov chain theory. In particular, we this paper, we describe two such adaptive algorithms, one show that much of the standard tool-set that is available for arising in a finite-horizon computation of expected reward the analysis of general state space chains is inapplicable to and the other arising in the context of solving eigenvalue the class of chains that a