A Hauptmann, “Meta-classification of multimedia classifiers.pdfVIP

Meta-classification of Multimedia Classifiers Wei-Hao Lin, Rong Jin, Alexander Hauptmann Language Technologies Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 USA {whlin,rong,alex}@ ABSTRACT Combining multiple classifiers is of particular interest in the multimedia systems, since there is usually data of very different types/modalities that should be mined or analyzed. Our wearable ‘experience collection’ system unobtrusively records the wearer’s conversation, recognizes the face of the dialog partner and remembers his/her voice. When the system sees the same person’s face or hears the same voice it can then use a summary of the last conversation with this person to remind the wearer. To correctly identify a person from a mixture of video and audio stream, classification judgments from individual modality classifiers must be combined effectively to yield a more accurate decision. A meta-classification strategy of combining multimodal classifiers using Support Vector Machine is proposed. Preliminary, empirical results show that combining different face recognition and speaker identification technology by meta-classification is dramatically more effective than weighted interpolation. Meta-classification is general enough to be applied to any application that needs to combine multiple classifiers without much modification. Keywords Meta-classification, Classifier Combination, Multimedia Classification, Face Recognition, Speaker Identification 1. INTRODUCTION Classification is an important form of knowledge extraction, and can help make key decisions. Multimedia classification works on visual or aural features that are quite different from text and numerical data, and needs to integrate results from algorithms in fields such as computer vision, pattern recognition, speech recognition, and machine learning. Visual classifiers for fingerprints [6], face recognition [11], [12], iris match