ONCONSISTENTFUSIONOFMULTIMODAL.PDFVIP

ON CONSISTENT FUSION OF MULTIMODAL BIOMETRICS S. Y. Kung Man-Wai Mak Dept. of Electrical Engineering Dept. of Electronic and Information Engineering Princeton University, USA The Hong Kong Polytechnic University, Hong Kong SAR ( A,V ) ( A,V ) ABSTRACT Fused Score s { s(X ) Audio-visual (AV) biometrics offer complementary informa- Adaptive Gating Network tion sources, and the use of both voice and facial images for bio- (e.g. hard-switch, linear combiner, and SVM) A A V V metric authentication has recently become economically feasible. s( ) { s(X ( ) ) s( ) { s(X ( ) ) Therefore, multi-modality adaptive fusion, combining audio and visual information, offers an ef?cient tool for substantially improv- Classifier for Classifier for ing the classi?cation performance. In terms of implementation, we Audio Channel Visual Channel propose to integrate an audio classi?er (based on Gaussian mixture ( A) models) and a visual classi?er (based on FaceIT, a commercially X X (V ) available software) into a well-established mixture-of-expert fu- sion architecture. In addition, a consistent fusion strategy is in- Fig. 1. Fusion network in an MOE (mixture-of-expert) architec- troduced as a baseline fusion scheme, which establishes the lower ture. Each vector sequence is compressed into a local score. The bound of the “consistent region” in the FAR-FRR ROC. O