Matching words and pictures：（匹配单词和图片）.pdfVIP

Journal of Machine Learning Research 3 (2003) 1107–1135 Submitted 4/02; Published 2/03 Matching Words and Pictures Kobus Barnard KOBU S@C S .ARIZONA .EDU Computer Science Department University of Arizona Tucson, AZ 85721-0077, USA Pinar Duygulu DUY GULU @CEN G .METU .EDU .TR Department of Computer Engineering Middle East Technical University Ankara, Turkey David Forsyth DAF @C S .BERKELEY.EDU Computer Science Division University of California Berkeley, CA 94720-1776, USA Nando de Freitas NAN DO @C S .UBC .CA Department of Computer Science University of British Columbia Vancouver, B.C. V6T 1Z4, Canada David M. Blei BLEI @C S .BERKELEY.EDU Computer Science Division University of California Berkeley, CA 94720-1776, USA Michael I. Jordan JORDAN @C S .BERKELEY.EDU Computer Science Division and Department of Statistics University of California Berkeley, CA 94720-1776, USA Editors: Jaz Kandola, Thomas Hofmann, Tomaso Poggio and John Shawe-Taylor Abstract We present a new approach for modeling multi-modal data sets, focusing on the specific case of segmented images with associated text. Learning the joint distribution of image regions and words has many applications. We consider in detail predicting words associated with whole images (auto-annotation) and corresponding to particular image regions (region naming). Auto-annotation might