KKTgeometry (从几何图形的角度来阐释KTT条件的意义).ppt

The Principles and Geometries of KKT and Optimization Geometries of KKT: Unconstrained Problem: Minimize f(x), where x is a vector that could have any values, positive or negative First Order Necessary Condition (min or max): ?f(x) = 0 (?f/?xi = 0 for all i) is the first order necessary condition for optimization Second Order Necessary Condition: ?2f(x) is positive semidefinite (PSD) [ x ? ?2f(x) ? x ≥ 0 for all x ] Second Order Sufficient Condition (Given FONC satisfied) ?2f(x) is positive definite (PD) [ x ? ?2f(x) ? x 0 for all x ] Geometries of KKT: Equality Constrained (one constraint) Problem: Minimize f(x), where x is a vector Subject to: h(x) = b First Order Necessary Condition for minimum (or for maximum): ?f(x) = ? ?h(x) for some ? free (? is a scalar) Two surfaces must be tangent h(x) = b and -h(x) = -b are the same; there is no sign restriction on ? Geometries of KKT: Equality Constrained (one constraint) First Order Necessary Condition: ?f(x) = ? ?h(x) for some ? Lagrangian: L(x, ?) = f(x) - ? [h(x)-b] , Minimize L(x, ?) over x and Maximize L(x, ?) over ?. Use principles of unconstrained optimization ?L(x, ?) = 0: ?xL(x, ?) = ?f(x) - ? ?h(x) = 0 ??L(x, ?) = h(x)-b = 0 Geometries of KKT: Equality Constrained (multiple constraints) Problem: Minimize f(x), where x is a vector Such that: hi(x) = bi for i = 1,2,…,m KKT Conditions (Necessary Conditions): Exist ?i ,i = 1,2,…,m, such that ?f(x) = ?i=1n ?i?hi(x) hi(x) = bi for i = 1,2,…,m Such a point (x, ?) is called a KKT point, and ? is called the Dual Vector or the Lagrange Multipliers. Furthermore, these conditions are sufficient if f(x) is convex and hi(x), i = 1,2,…,m, are linear. Geometries of KKT: Unconstrained, Except Non-Negativity Condition Problem: Minimize f(x), where x is a vector, x 0 First Order Necessary Condition: ?f/?xi = 0 if xi 0 ?f/?xi ≥ 0 if xi = 0 Thus: [?f/?xi]xi = 0 for all xi, or ?f(x) ? x = 0, ?f(x) ≥ 0 If in