神经网络课件.ppt

Network Architecture Single layer Multiple layer fully connected Recurrent network without hidden units inputs outputs Recurrent network with hidden units Unit delay operator Knowledge Representation Knowledge refers to stored information or models used by a person or machine to interpret, predict and appropriately respond to the outside world What information is actually made explicit; How the information is physically encoded for the subsequent use Good solution depends on good representation of knowledge In NN, knowledge is represented by internal network parameters real challenge Knowledge of the world world state represented by known facts - prior knowledge observations - obtained by (noisy) sensors; training examples Knowledge Acquisition by NN Training Training examples : either labeled or unlabeled labeled : input signal and desired response unlabeled : different realizations of input signal Examples represent the knowledge of environment Handwritten digit recognition 1. Appropriate architecture is selected for NN source node = number of pixels of input image 10 output node for each digit subset of examples for training NN by suitable learning algorithm 2. Recognition performance is tested by the rest of the examples Positive and negative examples Rules of Knwldg representation in NN 1. Similar input from similar classes produce similar representations similarity measures Euclidian distance, dot (inner) product, cos random variable : Mahalanobis distance ... 2. Separate classes produce widely different representations 3. More neurons should be involved in representation of more important feature probability of detection / false alarm 4. Prior information and invariances should be built into the design of the network general purpose vs specialized Building Prior to NN design Specialized structure learns fast because of small free parameters runs fast because of simple structure No well-defined rules for building specialized NN ad hoc approeach restricting