Newton-typemethods：牛顿型的方法.pdfVIP

Convergence analysis of inexact proximal Newton-type methods Jason D. Lee and Yuekai Sun Institute for Computational and Mathematical Engineering Stanford University, Stanford, CA jdl17,yuekai@stanford.edu Michael A. Saunders Department of Management Science and Engrineering Stanford University, Stanford, CA saunders@stanford.edu Abstract We study inexact proximal Newton-type methods to solve convex optimization problems in composite form: where is convex and continuously differentiable and is a convex but not necessarily differentiable function whose proximal mapping can be evaluated efficiently. Proximal Newton-type methods require the solution of subproblems to obtain the search directions, and these subproblems are usually solved using first-order methods or coordinate descent methods, which converge to the solu- tion linearly. In this paper, we analyze the convergence rate of inexact proximal Newton method, which solves the subproblems inexactly. 1 Introduction Many problems of relevance in machine learning, signal processing, and high dimensional statistics can be posed in composite form: (1) where is a convex, continuously differentiable loss function and is a convex, continuous, but not necessarily differentiable penalty function. Such problems include: (i) the lasso [1] (ii) multitask learning [2] and (iii) trace-norm matrix completi