best regression using information criteria（使用信息条件最好的回归）（14页）.docVIP

“Best Regression” Using Information Criteria Phill Gagne C. Mitchell Dayton University of Maryland 1. Introduction Statement of Problem Exploratory model building is often used within the context of multiple regression (MR) analysis. As noted by Draper Smith (1998), these undertakings are usually motivated by the contradictory goals of maximizing predictive efficiency and minimizing data collection/monitoring costs. A popular compromise has been to adopt some strategy for selecting a “best” subset of predictors. Many different definitions of “best” can be found in the applied literature including incremental procedures such as forward selection MR, backward elimination MR and stepwise MR and all-possible subsets MR with criteria related to residual variance, multiple correlation, Mallows Cp, etc. Incremental procedures are computationally efficient but do not, necessarily, result in the selection of an unconditionally “best” model. For example, as usually implemented, forward selection MR includes additional variables in the regression model based on maximizing the increment to R2 from step to step. At the third step, for example, the model contains the “best” three predictors only in a conditional sense. Also, the modifications to forward selection incorporated into stepwise MR, for example, do not guarantee finding the best three predictors. In contrast to incremental procedures, all-possible subsets does choose a “best” model for a fixed number of predictors but not necessarily an overall “best” model. For example, for the mth model based on pm out of a total of p independent parameters, Mallows CP utilizes a criterion of the form where is the residual variance estimate based on the full model (i. e., the model with all p predictors). Models with values close to pm + 1 are “best” in a final prediction error (FPE) sense. Thus, a “best” model can be identified for fixed values of pm but there is no general method for selecting an overall “best” model. The purpo