“Saliency-Based Visual Attention” .ppt

April 6, 2004 “Saliency-Based Visual Attention”  Overview Background System architecture The saliency map Preprocessing Feature maps Feature integration Focus of attention Results Conclusion Related Work “Feature Integration Theory,” Treisman Gelade, 1980. Computational model of bottom-up attention, Koch and Ullman, 1985 Saliency map is believed to be located in the posterior parietal cortex (Gotlieb, et al., 1998) and the pulvinar nuclei of the thalamus (Roinson Peterson, 1992) Architecture Gaussian Pyramids Repeated low-pass filtering =0, 1, 2, 3,…,8 I(0) is original input Preprocessing Original image with red, green, blue channels Intensity as I = (r + g + b)/3 Broadly tuned color channels R = r - (g + b)/2 G = g - (r + b)/2 B = b - (r + g)/2 Y = (r + g)/2 - |r – g|/2 - b Preprocessing Center-surround Difference Achieve center-surround difference through across-scale difference Operated denoted by Q: Interpolation to finer scale and point-to-point subtraction One pyramid for each channel: I(s), R(s), G(s), B(s), Y(s)where s ? [0..8] is the scale Intensity Feature Maps I(c, s) = | I(c) Q I(s)| c ? {2, 3, 4} s = c + d where d ? {3, 4} So I(2, 5) = | I(2) Q I(5)| I(2, 6) = | I(2) Q I(6)| I(3, 6) = | I(3) Q I(6)| … ? 6 Feature Maps Colour Feature Maps Similar to double-opponent cells (Prim. V. C) Red-Green and Yellow-Blue RG(c, s) = | (R(c) - G(c)) Q (G(s) - R(s)) | BY(c, s) = | (B(c) - Y(c)) Q (Y(s) - B(s)) | Same c and s as with intensity Orientation Feature Maps Create Gabor pyramids for q = {0o, 45o, 90o, 135o} c and s again similar to intensity Normalization Operator Promotes maps with few strong peaks Surpresses maps with many comparable peaks Normalization of map to range [0…M] Compute average m of all local maxima Find the global maximum M Multiply the map by (M – m)2 Normalization Operator Conspicuity Maps Saliency Map Average all conspicuity maps Neural Layers Saliency Map (SM) modeled as layer of le