Training Hierarchical Feed-forward Visual Recognition Models 培训层次前馈视觉识别模型.pptVIP

Overview of Back Propagation Algorithm Shuiwang Ji A Sample Network Forward Operation The general feed-forward operation is: Back Propagation Algorithm The hidden to output weights can be learned by minimizing the error The power of back-propagation is that it allows us to calculate an effective error for each hidden unit, and thus derive a learning rule for the input-to-hidden weights We consider the error function: The update rule is: Hidden-to-output Weights Input-to-hidden Weights Back Propagation of Sensitivity Training Hierarchical Feed-forward Visual Recognition Models Using Transfer Learning from Pseudo-Tasks ECCV’08 Kai Yu Presented by Shuiwang Ji Transfer Learning Transfer learning, also known as multi-task learning, is a mechanism that improves generalization by leveraging shared domain-specific information contained in related tasks In the setting considered in this paper, all tasks share the same input space General Formulation The main task to be learnt has index m with training examples A neural network has a natural architecture to tackle this learning problem by minimizing: General Formulation The is learned by additionally introducing pseudo auxiliary tasks, each represented by learning the input-output pairs: Then the regularization term becomes A Bayesian perspective (skipped) CNN for Transfer Learning Generating Pseudo Tasks Generating Pseudo Tasks Applying Gabor filters of 4 orientations and 16 scales result in 64 feature maps of size 104*104 for each image Max-pooling operation is performed first within each non-overlapping 4*4 neighborhood and then within each band of two successive scales resulting in 32 feature maps of size 26*26 for each image An set of K RBF filter of size 7*7 with 4 orientations are then sampled and used as the parameters of the pseudo-tasks, resulting in 8 feature maps of size 20*20 Finally, max pooling is performed on the result across all the scales and within every non-ov