flexible discriminant and mixture models.pdf

Stanford University—April 28, 1997 Flexible Discriminant and Mixture Models 1 $ Flexible Discriminant and Mixture Models Trevor Hastie trevor@ Statistics Department and Division of Biostatistics Stanford University joint with Andreas Buja and Rob Tibshirani April 28, 1997 Papers fda.ps.Z, pda.ps.Z and mda.ps.Z are available from: ftp://playfair.stanford/edu/pub/hastie % Stanford University—April 28, 1997 Flexible Discriminant and Mixture Models 2 $ Theme: Modular Extensions of Standard Tools Linear Discriminant Analysis or LDA is a classic technique for discrimination and classification Virtues of LDA: + Simple prototype method for multiple class classification + Can produce optimal low dimensional views of the data + Sometimes produces the best results; e.g. LDA featured in top 3 classifiers for 11/22 of the STATLOG datasets, overall winner in 3/22. Limitations of LDA: - Lots of data, many predictors: LDA underfits (restricts to linear boundaries) - Many correlated predictors: LDA (noisy/wiggly coefficients) - Dimension reduction limited by the number of classes % Stanford University—April 28, 1997 Flexible Discriminant and Mixture Models 3 $ Example of extension: FDA Y S Y where Y is an indicator response matrix