A knowledge-based approach to video content classification.pdf

A Knowledge-Based Approach to Video Content Classification? Y. Chen and E. K. Wong Department of Computer and Information Science Polytechnic University 5 Metrotech Center Brooklyn, NY 11201 Abstract A framework for video content classification using a knowledge-based approach is herein proposed. This approach is motivated by the fact that videos are rich in semantic contents, which can best be interpreted and analyzed by human experts. We demonstrate the concept by implementing a prototype video classification system using the rule-based programming language CLIPS 6.05. Knowledge for video classification is encoded as a set of rules in the rule base. The left-hand-sides of rules contain high level and low level features, while the right-hand-sides of rules contain intermediate results or conclusions. Our current implementation includes features computed from motion, color, and text extracted from video frames. Our current rule set allows us to classify input video into one of five classes: news, weather reporting, commercial, basketball and football. We use MYCIN’s inexact reasoning method for combining evidences, and to handle the uncertainties in the features and in the classification results. We obtained good results in a preliminary experiment, and it demonstrated the validity of the proposed approach. Keywords: video content classification, knowledge-based, rule-based, certainty factors, semantic representation 1. Introduction The use of low-level features, which is the approach used in most video retrieval and analysis systems today, has reached a limit in their capabilities. For many videos, human has a priori knowledge or expectations about their contents. This knowledge is acquired over time through experience in viewing many similar videos of the same type. For example, when viewing the video of a basketball game, the expectations include, among others, fast motion, orange (or other) color basketball, a wood color floor, periodic displays of game scores o