电子讲义-5离散时间马尔科夫链.pdfVIP

(A.A.Markov)  t  t  t  (),  Æ ÆÆ Æ ℄ §5.1 ℄ §5.2 §5.3 §5.4 §5.1 ℄ T = {0, 1, 2, }, S = {0, 1, 2, } 5.1.1 ξ = {ξn , n = 0, 1, 2, } S n ≥ 0 i , i , i , , i , i ∈ S 0 1 2 n n+1 P (ξn+1 = in+1 | ξk = ik , 0 ≤ k ≤ n) = P (ξn+1 = in+1 | ξn = in ) (5.1.1) ξ (discrete-time Markov chain) (5.1.1) p (n; i, j) = P (ξn+1 = j | ξn = i), ∀ i, j ∈ S 182 ξ n (one-step transition probability), P (n) = (p (n; i, j))i,j ∈S ξ n (one-step transition probability matrix). Æ℄ () n (temporally inho- mogeneous). s i s + t ℄ j (s = 0) i t ℄ j . ℄ 5.1.2 ξ = {ξn , n = 0, 1, 2, } ℄ p (n; i, j), i, j ∈ S n ξ (temporally homogeneous Markov chain). Æ℄ 5.1.2, pij = P (ξn+1 = j | ξn = i), pij (n) = P (ξn = j | ξ0 = i), i, j ∈ S. (5.1.2) P (n) = (pij (n))i,j ∈S n (n-steps transition probability matrix). p (0) = P (ξ = i), i ∈ S (5.1.3) i 0 ξ (initial distribution); p (n) = P (ξ = i), i ∈ S (5.1.4) i n ξ n Æ(absolute probability distribution). ℄ i, j ∈ S ,   1, i = j ; pij (0) =  0, i = j. Æ℄ 5.1.3 ξ = {ξn , n = 0, 1, 2, }