优化报告.pptVIP

Optimization Problem 齐娟 2007-11-4 “in fact, the great watershed in optimization isnt between linearity and nonlinearity, but convexity and nonconvexity R. Tyrrell Rockafellar, 1993 whether there is a feasible solution ? finding the globally optimal solution? solving the problem up to very large size ? Optimality criterion for differentiable f0 Optimality criterion for differentiable f0--unconstrained Interior-point method | barrier method In a few special cases, we can find the derivatives easily, but usually the problem must be solved by an iterative algorithm. Idea: Application: Linear Regression , Curve-fitting Constrained Least-squares Problem Lagrange method: Linear SVM Mathematically What we know: w . x+ + b = +1 w . x- + b = -1 w . (x+-x-) = 2 The solution has the form: Then the classifying function will have the form: QP package (MINOS,LOQO,MATLAB toolbox etc.) MATLAB优化工具箱能求解的优化模型  one of the first optimization problems to be studied back in the 1930s and 40s. It was first motivated by the Armys desire to meet the nutritional requirements of the field GIs while minimizing the cost. The linear program consisted of nine equation in 77 unknowns. It took nine clerks using hand-operated desk calculators 120 man days to solve for the optimal solution Loss: (Eigenvalue problem) Solution: smallest eigenvalue Constrained Least-squares Problem More general: Generalized eigenvalue problem QZ algorithm Example Ⅱ:SVM f x a yest denotes +1 denotes -1 f(x,w,b) = sign(w x + b) Any of these would be fine.. ..but which is best? Maximum Margin f x a yest denotes +1 denotes -1 f(x,w,b) = sign(w x + b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM) Linear SVM Support Vectors are those datapoints that the margin pushes up against Maximizing the margin is good according to