Modelselection.pptVIP

Model selection Stepwise regression Statement of problem A common problem is that there is a large set of candidate predictor variables. (Note: The examples herein are really not that large.) Goal is to choose a small subset from the larger set so that the resulting regression model is simple, yet have good predictive ability. Example: Cement data Response y: heat evolved in calories during hardening of cement on a per gram basis Predictor x1: % of tricalcium aluminate Predictor x2: % of tricalcium silicate Predictor x3: % of tetracalcium alumino ferrite Predictor x4: % of dicalcium silicate Example: Cement data Two basic methods of selecting predictors Stepwise regression: Enter and remove predictors, in a stepwise manner, until there is no justifiable reason to enter or remove more. Best subsets regression: Select the subset of predictors that do the best at meeting some well-defined objective criterion. Stepwise regression: the idea Start with no predictors in the “stepwise model.” At each step, enter or remove a predictor based on partial F-tests (that is, the t-tests). Stop when no more predictors can be justifiably entered or removed from the stepwise model. Stepwise regression: Preliminary steps Specify an Alpha-to-Enter (αE = 0.15) significance level. Specify an Alpha-to-Remove (αR = 0.15) significance level. Stepwise regression: Step #1 Fit each of the one-predictor models, that is, regress y on x1, regress y on x2, … regress y on xp-1. The first predictor put in the stepwise model is the predictor that has the smallest t-test P-value (below αE = 0.15). If no P-value 0.15, stop. Stepwise regression:Step #2 Suppose x1 was the “best” one predictor. Fit each of the two-predictor models with x1 in the model, that is, regress y on (x1, x2), regress y on (x1, x3), …, and y on (x1, xp-1). The second predictor put in stepwise model is the predictor that has the smallest t-test P-value (below αE = 0.15). If no P-value 0.15, stop. Stepwise regression:Step