Communication and regeneration Heuristics推荐.pdfVIP

Chapter 1 Heuristics This book is about Markovian models, and particularly about the structure and stability of such models. We develop a theoretical basis by studying Markov chains in very general contexts; and we develop, as systematically as we can, the applications of this theory to applied models in systems engineering, in operations research, and in time series. A Markov chain is, for us, a collection of random variables Φ = {Φn : n ∈ T }, where T is a countable time set. It is customary to write T as Z+ := {0, 1, . . .}, and we will do this henceforth. Heuristically, the critical aspect of a Markov model, as opposed to any other set of random variables, is that it is forgetful of all but its most immediate past. The precise meaning of this requirement for the evolution of a Markov model in time, that the future of the process is independent of the past given only its present value, and the construction of such a model in a rigorous way, is taken up in Chapter 3. Until then it is enough to indicate that for a process Φ, evolving on a space X and governed by an overall probability law P, to be a time-homogeneous Markov chain, there must be a set of “transition probabilities” {P n (x, A), x ∈ X, A ⊂ X} for appropriate sets A such that for times n, m in Z+ P(Φ ∈ A | Φ , j ≤ m; Φ = x) = P n (x, A); (1.1) n +m j m that is, P n (x, A) denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of P n on the values of Φ , j ≤ m, is the Markov j property, and the independence of P n and m is the time-homogeneity property. We now show that systems which are amenable to modeling by discrete time Markov