酩悦潒敢-ProfitingfromDataMining.pptVIP

Profiting from Data Mining Bob Stine Department of Statistics The Wharton School, Univ of Pennsylvania April 5, 2002 /~bob Overview Critical stages of data mining process Choosing the right data, people, and problems Modeling Validation Automated modeling Feature creation and selection Exploiting expert knowledge, “insights” Applications Little detail – Biomedical: finding predictive risk factors More detail – Financial: predicting returns on the market Lots of detail – Credit: anticipating the onset of bankruptcy Predicting Health Risk Who is at risk for a disease? Example: detect osteoporosis without expense of x-ray Goals Improving public health Savings on medical care Confirm an informal model with data mining Many types of features, interested groups Clinical observations of doctors Laboratory measurements, “genetic” Self-reported behavior Missing data Predicting the Stock Market Small, “hands-on” example Goals Better retirement savings? Money for that special vacation? College? Trade-offs: risk vs return Lots of “free” data Access to accurate historical time trends, macro factors Recent data more useful than older data “Simple” modeling technique Validation Predicting the Market: Specifics Build a regression model Response is return on the value-weighted SP Use standard forward/backward stepwise Battery of 12 predictors with interactions Train the model during 1992-1996 (training data) Model captures most of variation in 5 years of returns Retain only the most significant features (Bonferroni) Predict returns in 1997 (validation data) Another version in Foster, Stine Waterman Historical patterns? Fitted model predicts... What happened? Claimed versus Actual Error Over-confidence? Over-fitting Model fits the training data too well – better than it can predict the future. Greedy fitting procedure “Optimization capitalizes on chance” Some intuition Coincidences Cancer clusters, the “birthday problem” Illustration with an auction What is the va