statinary distribution of Markov chain.pdf

The Stationary Distribution of a Markov Chain Alessandro Panconesi DI, La Sapienza of Rome May 15, 2005 In this note I present a concise proof of the existence and uniqueness of the limit distribution of an ergodic markov chain. This nice proof was described to me by David Gilat of Hebrew University during a hot summer afternoon in Perugia. A finite n ×n transition probability matrix P := [pij ] is a stochastic matrix where pij is the transition probability of going from state i to state j . We can think of a directed graph whose edges are weighted with transition probabilities. Note that since P is stochastic, the sum of the probabilities of the arcs outgoing from a vertex i sum up to 1, i.e. j pij = 1. We are interested in keeping track of the random walk of a pebble. The pebble is initially placed on the graph according to an initial distribution X0 and then it proceeds by traversing the edges ij ’s with their associated probabilities pij ’s. X0 is a probability distribution, Xi being the probability of placing the pebble on vertex 0 i initially. After one step the position of the pebble is given by the probability distribution X = X P 1 0 and in general after t steps we have t X = X P = X P . t t−1 0 The infinite sequence X , X , X , . . . is called a markov chain. In what follows we shall 0 1 2 use the term markov chain somewha