Caption analysis and recognition for building video indexing systems. Multimedia systems.pdf

1 Caption Analysis and Recognition For Building Video Indexing Systems Fu Chang?, Guey-Ching Chen?, Chin-Chin Lin?? and Wen-Hsiung Lin? ?Institute of Information Science, Academia Sinica ?Dept. of Electrical Engineering, National Taipei University of Technology, Taipei, Taiwan E-Mail: {fchang, ching64, erikson, bowler}@.tw ABSTRACT In this paper, we propose several methods for analyzing and recognizing Chinese video captions, which constitute a very useful information source for video content. Image binarization, performed by combining a global threshold method and a win- dow-based method, is used to obtain clearer images of characters; and a cap- tion-tracking scheme is used to locate caption regions and detect caption changes. The separation of characters from possibly complex backgrounds is achieved by us- ing size and color constraints, and by cross-examination of multi-frame images. To segment individual characters, we use a dynamic split and merge strategy. Finally, we propose a character recognition process using a prototype classification method, supplemented by a disambiguation process using support vector machines, to im- prove recognition outcomes. This is followed by a post-process that integrates mul- tiple recognition results. The overall accuracy rate for the entire process applied to test video films is 94.11%. Key Words: background removal, caption tracking, character segmentation, char- acter recognition, image binarization, support vector machines, prototype classifica- tion 1 Introduction The rapid increase in the use of digital videos in recent years has raised the need for an archival storage system. Such a system, in turn, requires an effective indexing technique for 2 retrieving and browsing video content (Aslandogan and Yu 1999). Since video captions are rich sources of information, caption-based retrieval has become a popular focus for research into video content retrieval. The aim of this paper is to provide systemat