多元LMChapter.pptVIP

Penalized Least Square * Some Famous Penalty Function * Properties of Lasso Estimator 1. Unbiasedness: nearly unbiased when the true unknown parameter is large to avoid unnecessary modeling bias. 2. Sparsity: Automatically sets small estimated coefficients to zero to reduce model complexity. 3. Continuity: the resulting estimator is continuous in data to avoid instability in model prediction. * Properties of Lasso Under some regularity conditions, we have (1) Sparsity: (2) Asymptotic normality: * LQA Algorithm Newton-Raphson algorithm: Step 1: Set an initial value: * Step 2: * Step 3: Set Repeat Steps 2 and 3 until convergence. * Determination of tuning parameter * Determination * Simulation Studies * Simulation Studies We simulated the data for B=1000 times, and get 1000 Lasso estimators Also we get 1000 Stepwise estimator, AIC estimator, BIC estimators. * Coparison * Table Estimator Correct No. InCorrec t No. Lasso AIC BIC Stepwise * * 变量选择 (模型选择） * * * * * * * * * 变量选择 * * * * * * * * AICc For small sample: * QAIC Quasi-AIC: * BIC * Stepwise Selection Method Forward Selection Backward Selection * Special Issue: Lasso(Least absolute Shrinkage and Selection Operator) * Model and Motivation Consider the simple constant mean model: Our Aim: Estimate the parameter and * Why it works Under the least square principle: * Why it works Usually, * What’s Lasso What’s wrong, it must be the least square principle. * * * What’s Lasso Consider the general linear model: * What’s Lasso *