MONOLINGUAL AND CROSSLINGUAL COMPARISON OF TANDEM FEATURES DERIVED FROM ARTICULATORY AND PH.pdf

MONOLINGUAL AND CROSSLINGUAL COMPARISON OF TANDEM FEATURES DERIVED FROM ARTICULATORY AND PHONE MLPS O?zgu?r C?etin 1 Mathew Magimai-Doss 2 Karen Livescu 3 Arthur Kantor 4 Simon King 5 Chris Bartels 6 Joe Frankel 5 1 Yahoo!, Inc., Santa Clara, USA 2 IDIAP Research Institute, Martigny, Switzerland 3 Massachusetts Institute of Technology, Cambridge, USA 4 University of Illinois, Urbana-Champaign, USA 5 University of Edinburgh, Edinburgh, UK 6 University of Washington, Seattle, USA ABSTRACT In recent years, the features derived from posteriors of a multilayer perceptron (MLP), known as tandem features, have proven to be very effective for automatic speech recognition. Most tandem features to date have relied on MLPs trained for phone classification. We recently showed on a relatively small data set that MLPs trained for articulatory feature clas- sification can be equally effective. In this paper, we provide a similar comparison using MLPs trained on a much larger data set—2000 hours of English conversational telephone speech. We also explore how portable phone- and articulatory feature- based tandem features are in an entirely different language— Mandarin—without any retraining. We find that while the phone-based features perform slightly better in the matched- language condition, they perform significantly better in the cross-language condition. Yet, in the cross-language condi- tion, neither approach is as effective as the tandem features extracted from an MLP trained on a relatively small amount of in-domain data. Beyond feature concatenation, we also explore novel observation modeling schemes that allow for greater flexibility in combining the tandem and standard fea- tures at hidden Markov model (HMM) outputs. Index Terms— Speech recognition, feedforward neural networks, hidden Markov models. 1. INTRODUCTION The so-called tandem acoustic modeling refers to a data- driven feature extraction method using MLPs [1, 2, 3]. In tandem modeling, the transformed posterior