Simultaneous Image Classification and Annotation同时图像分类与注释.pptVIP

Simultaneous Image Classification and Annotation Chong Wang, David Blei, Li Fei-Fei Computer Science Department Princeton University Published in CVPR 2009 Presented by Eric Wang 7-3-09 (Final Version) Outline Introduction Review of sLDA and Corr-LDA Model description Model Inference and Parameter Estimation Empirical Results Conclusion Introduction Images Classification refers to assigning a class label to each image which globally describes the image. Image Annotation refers to assigning words which describe individual regions of the image. The images considered in this paper are both classified with a class label and annotated with free text. This paper will combine the basic framework of Corr-LDA, and a highly modified version of Supervised LDA (sLDA) to yield a model which simultaneously classifies images and annotates the individual regions. Review of sLDA For each document In this model, the response variable y is a continuous random variable. are treated as unknown constants to be estimated, rather than as random variables. Source: D. M. Blei and J. D. McAuliffe. Supervised topic models. In NIPS, 2007. Review of sLDA An application of sLDA considered by Blei et. al was regressing a corpus of textual movie reviews to number of stars given. Source: D. M. Blei and J. D. McAuliffe. Supervised topic models. In NIPS, 2007. Review of Corr-LDA For each document Corr-LDA is a simple extension of LDA for images to annotate image regions. Source: D. M. Blei and M. I. Jordan. Modeling annotated data. In SIGIR, 2003. Annotated sLDA Model This step is identical to LDA, the topic proportions are drawn once per document. In this paper, is not optimized. Annotated sLDA Model A region is characterized by one of 240 codewords (quantized from 128 dimensional SIFT features). Regions are found by segmenting images using the N-cuts algorithm. parameterizes a particular multinomial distribution (topic) over the