NumberofVisitstoaState.docVIP

2.7 Number of Visits to a State 2.7.1 Number of visits to a state In section 2.6.1 we calculated the probability of reaching a state. In this section we want to calculate the expected number of visits to a state in the case where this is not infinite. As an example we use the circuit board manufacturing example used in section 2.6.1. Example 1. A company manufactures circuit boards. There are four steps involved. 1. Tinning 2. Forming 3. Insertion 4. Soldering There is a 5% chance that after the forming step a board has to go back to timing. There is a 20% chance that after the insertion step the board has to be scrapped. There is a 30% chance that after the soldering step the board has to go back to the insertion step and a 10% chance that the board has to be scrapped. What is the expected number of visits to each of the states in the production of one good board. If a board is scrapped then we start over with tinning and include the visits to each of the states for the scrapped board. We make a small modification to the transition diagram used in the previous section. This time we go from the scrapped state (5) back to the tinning state (1). The transition matrix is now (1) P = The number of visits to any particular state in the production of a good board is a random variable, since the number of visits will vary with the particular board being produced. Definition 1. If i and j are states in a Markov chain, let (2) N(j) = the total number of times that the system is in state j (3) sij = E{N(j) | X0 = i} = expected number of visits to state j if we start in state i (4) S = (5) ST = restriction of S to the transient states, i.e. we only include rows and columns corresponding to the transient states where m is the total number of states. If the system is in state j at time n = 0, then we include this in the count of the total number of times. N(j) is a random variable that depends on the values of X0, X1, … It is possib