The Group Lasso for Logistic Regression.ppt

* The Group Lasso for Logistic Regression Lukas Meier, Sara van de Geer and Peter Bühlmann Presenter: Lu Ren ECE Dept., Duke University Sept. 19, 2008 Outline From lasso to group lasso logistic group lasso Algorithms for the logistic group lasso Logistic group lasso-ridge hybrid Simulation and application to splice site detection Discussion Lasso A popular model selection and shrinkage estimation method. In a linear regression set-up: :continuous response :design matrix : parameter vector The lasso estimator is then defined as: where , and larger set some exactly to 0. Group Lasso In some cases not only continuous but also categorical predictors (factors) are present, the lasso solution is not satisfactory with only selecting individual dummy variables but the whole factor. Extended from the lasso penalty, the group lasso estimator is: : the index set belonging to the th group of variables. The penalty does the variable selection at the group level , belonging to the intermediate between and type penalty. It encourages that either or for all Consider a case: two factors and Observe the contour of the penalty function: -penalty treats the three co-ordinate directions differently: encourage sparsity in individual coefficients while -penalty treats all directions equally and does not encourage sparsity. Connection Ref: Ming Yuan and Yi Lin, Model selection and estimation in regression with grouped variables, J.R. Statist.,2008 Logistic Group Lasso Independent and identically distributed observations : p-dimensional vector of predictors : a binary response variable, : feedom degree The conditional probability with The estimator is given by the minimizer of the convex function: Logistic Group Lasso controls the amount of penalization rescale t