Communication and regeneration Markov models推荐.pdfVIP

Chapter 2 Markov models The results presented in this book have been written in the desire that practitioners will use them. We have tried therefore to illustrate the use of the theory in a systematic and accessible way, and so this book concentrates not only on the theory of general space Markov chains, but on the application of that theory in considerable detail. We will apply the results which we develop across a range of specific applications: typically, after developing a theoretical construct, we apply it to models of increas- ing complexity in the areas of systems and control theory, both linear and nonlinear, both scalar and vector valued; traditional “applied probability” or operations research models, such as random walks, storage and queueing models, and other regenerative schemes; and models which are in both domains, such as classical and recent time series models. These are not given merely as “examples” of the theory: in many cases, the appli- cation is difficult and deep of itself, whilst applications across such a diversity of areas have often driven the definition of general properties and the links between them. Our goal has been to develop the analysis of applications on a step-by-step basis as the theory becomes richer throughout the book. To motivate the general concepts, then, and to introduce the various areas of appli- cation, we leave until Chapter 3 the normal and necessary foundations of the subject, and first introduce a cross-section of the models for which we shall be developing those foundations. These models are still described in a somewhat heuristic way. The full mathematical description of their dynamics must await the development in the next chapter of the concepts of transition probabilities, and the reader may on occasion benefit by moving to some of those descriptions in parallel with the outlines