TreeBasedMethods.pptxVIP

Tree Based Methods Longfei Wang 2016.10.14 Content The basics of decision trees Regression trees Classification trees Trees versus linear models Advantages and disadvantages of trees Bagging, random forests, boosting Bagging Random forests boosting Lab: decision trees Fitting classification trees Fitting regression trees Bagging and random forests boosting 8.1.1 Regression Trees Internal node branch terminal node /leaf Mean response value 8.1.1 Regression Trees 8.1.1 Regression Trees Top Left: A partition of two-dimensional feature space that could not result from recursive binary splitting. Top Right: The output of recursive binary splitting on a two-dimensional example. Bottom Left: A tree corresponding to the partition in the top right panel. Bottom Right: A perspective plot of the prediction surface corresponding to that tree. 8.1.1 Regression Trees 8.1.1 Regression Trees 8.1.1 Regression Trees 8.1.2 Classification Trees 8.1.2 Classification Trees Predicting Baseball Players’ Salaries Data: Heart, contain a binary outcome HD for 303 patients who presented with chest pain. 8.1.3 Trees Versus Linear Models 8.1.4 Advantages Disadvantages of Trees Advantages Trees are very easy to explain. Decision trees more closely mirror human decision-making than classical approaches. Trees can be displayed graphically and are easily interpreted. Trees can easily handle qualitative predictors without the need to create dummy variables. Disadvantages Trees generally do not have the same level of predictive accuracy as the classical approaches. Trees can be very non-robust. ? By aggregating many decision trees, using methods like bagging, random forest, and boosting, the performance can be improved. 8.2.1 Bagging 8.2.2 Random Forest 8.2.3 Boosting