WHY MEANINGFUL AUTOMATIC TAGGING OF IMAGES IS VERY HARD Theo为什么有意义的图像自动标注是非常困难的西奥.pptVIP

May 2009 ICME2009 talk WHY MEANINGFUL AUTOMATIC TAGGING OF IMAGES IS VERY HARD Theo Pavlidis? Stony Brook University t.pavlidis@ieee.org We expect that dealing with images to be much harder than dealing with text. The human visual system has evolved from animal visual systems over a period of more than 200 million years. Three Specific Pieces of Evidence why Auto-Tagging is hard Failure of past pixel-based techniques to scale to real world data. Efforts to base tagging on non-pixel information and their limits. Security systems based on the assumption that automatic tagging is impossible. Pixel-based methods do not scale Methods work well in published examples, but fail at large because of: Huge cardinality of the set of all possible images: the number of different discernible images is at least 1025 (over a trillion squared). Semantic gap (actually semantic abyss) A pair from a set of 536 (1025) images Cardinality Problems Because the number of images is so large it is very hard to find a representative sample. Even if many of the different images may have the “same” meaning for a human viewer, their pixel values may differ a lot. Hence the semantic and other gaps. Aside: The cardinality problem can be dealt by limiting the class of images and the matching rules (examples are applications in biometrics). Using synthetic data (if we know the rules) also helps. The Semantic Abyss The Conceptual Abyss A Major Obstacle Human observers tend to agree on images that are quite similar or quite dissimilar (slide on “semantic abyss”) but not on those in between (slide on “conceptual abyss”). If there is no agreement on similarity amongst human observers how can we establish computational measures for similarity? Tagging (Labeling) is much harder than matching because it requires interpretation Results from ALIPR Result No. 1 from a new site Result No. 2 from a new site Three Specific Pieces of Evidence why Auto-Tagging is hard Failure of past pixel-based techniques to scal