5 State Machines.pptVIP

5 State Machines Copyright ? Department of Computer Science, Guizhou University 5 State Machines Representing the Environment by Feature Vectors Elman networks Iconic representations Backboard systems 5.1 Representing the Environment by Feature Vectors Motivation Sensory limitations of the agent preclude completely accurate representation of environmental state by feature vectors. The accuracy can be improved by taking into account previous sensory history. 状态机 Xt=F(St,Xt-1,at-1) 其中，Xt是t时刻的特征向量；St是t时刻的感知输入； Xt-1是t-1时刻的特征向量；at-1是t-1时刻采取的动作。 Cont’d A somewhat sensory-impaired version of the boundary-following robot Only 4 sensory inputs (s2,s4,s6,s8) are sensed. Even with this impairment, this robot can still perform boundary-following behavior if it computes the needed feature vector from its immediate sensory inputs, the previous feature vector, and the just-performed action. wi=si, for i=2,4,6,8 w1=1 iff at the previous time step w2 had value 1 and the robot moved east. Cont’d Similar for w3, w5, w7 Cont’d The following production system gives wall-following behavior: w2?w4?east x1?x2?east w4?w6?south x2?x3?south w6?w8?west x3?x4?west w8?w2?north x4?x1?north w1?north x4?x1?north w3?east x1?x2?east w5?south x2?x3?south w7?west x3?x4?west 1?north 1?north Cont’d Conclusion Sensory abilities are always limited in some way, and thus agents equipped with stored models of the environment will usually be able to perform tasks that memory-less agents cannot. 5.2 Elman Networks Elman network a special type of recurrent neural network. For the boundary-following robot Inputs: (s2, s4, s6, s8) + the values of the eight hidden units one time step earlier (context units ) Hidden units: eight hidden units, one for each feature Outputs: four output units, one for each action See Figure 5.2 5.3 Iconic Representations（直观表示） Iconic representation /Analogical representation simulations of important aspects of the environment. Sensory information is us