Data Mining Casualty Actuarial Society数据挖掘意外保险精算学会.ppt

Data Mining CAS 2004 Ratemaking Seminar Philadelphia, Pa. Louise Francis, FCAS, MAAA Louise_francis@ Objectives Answer the question: Why use data mining? Introduce the main data mining methods Decision Trees Neural Networks MARS Clustering The Data Simulated Data for Automobile Claim Frequency Three Factors Territory Four Territories Age Continuous Function Mileage High, Low Data Challenges Nonlinearities Relation between dependent variable and independent variables is not linear or cannot be transformed to linear Interactions Relation between independent and dependent variable varies by one or more other variables Correlations Predictor variables are correlated with each other Simulated Example: Probability of Claim vs. Ageby Territory and Mileage Group Claim Frequency Data Independent Probabilities for Each Variable Decision Trees Recursively partitions the data Often sequentially bifurcates the data – but can split into more groups Applies goodness of fit to select best partition at each step Selects the partition which results in largest improvement to goodness of fit statistic Goodness of Fit Statistics Chi Square ? CHAID (Fish, Gallagher, Monroe- Discussion Paper Program, 1990) Deviance ? CART Goodness of Fit Statistics Gini Measure ? CART Goodness of Fit Statistics Entropy ? C4.5 First Split Example of Goodness of Fit Calculation Example of Fitted Tree MARS Multivariate Adaptive Regression Splines An extension of regression which Uses automated search procedures Models nonlinearities Models interactions Produces a regression-like formula Nonlinear Relationships Fits piecewise regression to continuous variables Interactions Fits basis functions (which are like dummy variables) to model interactions An interaction between Territory=East and Mileage can be modeled by a dummy variable which is 1 if the Territory=East and mileage =High and 0 otherwise. Goodness of Fit Statistics Generalized Cross-Validation Fitted MARS Model Neural Networks Developed