深度学习稀疏编码教程.ppt

MNIST results -- learned dictionary * A hidden unit in the second layer is connected to a unit group in the 1st layer: invariance to translation, rotation, and deformation Yu, Lin, Lafferty, CVPR 11 Caltech101 results - classification ? Learned descriptor: performs slightly better than SIFT + SC * Yu, Lin, Lafferty, CVPR 11 Adaptive Deconvolutional Networks for Mid and High Level Feature Learning Hierarchical Convolutional Sparse Coding. Trained with respect to image from all layers (L1-L4). Pooling both spatially and amongst features. Learns invariant mid-level features. Matthew D. Zeiler, Graham W. Taylor, and Rob Fergus, ICCV 2011 Select L2 Feature Groups Select L3 Feature Groups Select L4 Features L1 Feature Maps Image L2 Feature Maps L4 Feature Maps L1 Features L3 Feature Maps Outline Sparse coding for image classification Understanding sparse coding Hierarchical sparse coding Other topics: e.g. structured model, scale-up, discriminative training Summary * * Other topics of sparse coding Structured sparse coding, for example Group sparse coding [Bengio et al, NIPS 09] Learning hierarchical dictionary [Jenatton, Mairal et al, 2010] Scale-up sparse coding, for example Feature-sign algorithm [Lee et al, NIPS 07] Feed-forward approximation [Gregor LeCun, ICML 10] Online dictionary learning [Mairal et al, ICML 2009] Discriminative training, for example Backprop algorithms [Bradley Jbagnell, NIPS 08; Yang et al. CVPR 10] Supervised dictionary training [Mairal et al, NIPS08] * * Summary of Sparse Coding Sparse coding is an effect way for (unsupervised) feature learning A building block for deep models Sparse coding and its local variants (LCC, SVC) have pushed the boundary of accuracies on Caltech101, PASCAL VOC, ImageNet, … Challenge: discriminative training is not straightforward * * * * * * Let’s further check what’s happening when best classification performance is achieved. * * * * * Hi I’m here to talk about my poster “Adaptive Deconvolution