Discussion of least angle regression.pdfVIP

Discussion of Least Angle Regression David Madigan Rutgers University Avaya Labs Research Piscataway, NJ 08855 madigan@ Greg Ridgeway RAND Statistics Group Santa Monica, CA 90407-2138 gregr@ Algorithms for simultaneous shrinkage and selection in regression and classifi- cation provide attractive solutions to knotty old statistical challenges. Nevertheless, as far as we can tell, Tibshirani’s Lasso algorithm has had little impact on statistical practice. Two particular reasons for this may be the relative inefficiency of the orig- inal Lasso algorithm, and the relative complexity of more recent Lasso algorithms (e.g., Osborne et al., 2000). Efron, Hastie, Johnstone, and Tibshirani have provided an efficient, simple algorithm for the Lasso as well as algorithms for stagewise- regression and the new least angle regression. As such this paper is an important contribution to statistical computing. Predictive Performance The authors say little about predictive performance issues. In our work, however, the relative out-of-sample predictive performance of LARS, Lasso, and forward- stagewise (and variants thereof) takes center stage. Interesting connections exist between boosting and stagewise algorithms so predictive comparisons with boost- ing are also of interest. The authors present a simple
statistic for LARS. In practice, a cross-validation type approach for selecting the degree of shrinkage, while computationally more expensive, may lead to better predictions. We considered this using the LARS soft- ware. Here we report results for the authors’ diabetes data, the Boston housing data, and the Servo data from the UCI Machine Learning Repositor