模型估计与选择.pptVIP

THE END of Talk * CART CART Example: Let’s consider a regression problem: continuous response Y inputs X1 and X2. To simplify matters, we consider the partition shown by the top right panel of figure. The corresponding regression model predicts Y with a constant Cm in region Rm： For illustration, we choose C1=-5, C2=-7, C3=0,C4=2, C5=4 in the bottom right panel in figure 9.2. Regression Tree Suppose we have a partition into M regions: R1 R2 …..RM. We model the response Y with a constant Cm in each region: If we adopt as our criterion minimization of RSS, it is easy to see that: Regression Tree(cont.) Finding the best binary partition in term of minimum RSS is computationally infeasible A greedy algorithm is used. Here Regression Tree(cont.) For any choice j and s, the inner minimization is solved by: For each splitting variable Xj the determination of split point s can be done very quickly and hence by scanning through all of the input, determination of the best pair (j,s) is feasible. Having found the best split, we partition the data into two regions and repeat the splitting progress in each of the two regions. Regression Tree(cont.) We index terminal nodes by m, with node m representing region Rm. Let |T| denotes the number of terminal notes in T. Letting: We define the cost complexity criterion: Classification Tree : the proportion of class k on mode : the majority class on node m Instead of Qm(T) defined in(9.15) in regression, we have different measures Qm(T) of node impurity including the following: Misclassification Error: Gini Index: Cross-entropy (deviance): Classification Tree(cont.) Example: For two classes, if p is the proportion of in the second class, these three measures are: 1-max(p,1-p) , 2p(1-p), -plog(p) -(1-p)log(1-p） * Additive Models * * Additive Models * Introduction 9.1: Generalized Additive Models 9.2: Tree-B