Markov-chains-马尔科夫链.docVIP

Markov Chains 4.1 INTRODUCTION AND EXAMPLES Consider a stochastic process {Xn,n=0,1,2, ...} that takes on a finite or countable number of possible values. Unless otherwise mentioned, this set of possible will be denoted by the set of nonnegative integers {0,1,2, ...}.If Xn=i, then the process is said to be in state i at time n. We suppose that whenever the process is in state i, there is a fixed probability Pij that it will next be in state j. That is, we suppose that (4.1.1) P{Xn+1=j Xn-1=in-1,...,X1=i1,X0=i0}=Pij for all states i0 ,i1,...in-1 , i, j and all n ≥0.Such a stochastic proc