Communication and regeneration Transition probabilities推荐.pdfVIP

Chapter 3 Transition probabilities As with all stochastic processes, there are two directions from which to approach the formal definition of a Markov chain. The first is via the process itself, by constructing (perhaps by heuristic arguments at first, as in the descriptions in Chapter 2) the sample path behavior and the dynamics of movement in time through the state space on which the chain lives. In some of our examples, such as models for queueing processes or models for controlled stochastic systems, this is the approach taken. From this structural definition of a Markov chain, we can then proceed to define the probability laws governing the evolution of the chain. The second approach is via those very probability laws. We define them to have the structure appropriate to a Markov chain, and then we must show that there is indeed a process, properly defined, which is described by the probability laws initially constructed. In effect, this is what we have done with the forward recurrence time chain in Section 2.4.1. From a practitioner’s viewpoint there may be little difference between the ap- proaches. In many books on stochastic processes, such as C¸ inlar [59] or Karlin and Taylor [194], the two approaches are used, as they usually can be, almost interchange- ably; and advanced monographs such as Nummelin [303] also often assume some of the foundational aspects touched on here to be well understood. Since one of our goals in this book is to provide a guide to modern general space Markov chain theory and methods for practitioners, we give in this chapter only a sketch of the full mathematical construction which provides the underpinning of Markov chain theory. However, we also have as another, and perhaps somewhat contradictory, goal the provision of a thorough and rigorous exposition of results