最优化理论课件.pptVIP

8 * Performance Surfaces Taylor Series Expansion Example Taylor series approximations: Taylor series of F(x) about x* = 0 : Plot of Approximations Vector Case F x ( ) F x * ( ) x 1 D D F x ( ) x x * = x 1 x 1 * – ( ) x 2 D D F x ( ) x x * = x 2 x 2 * – ( ) + + = … x n D D F x ( ) x x * = x n x n * – ( ) 1 2 - - - x 1 2 2 D D F x ( ) x x * = x 1 x 1 * – ( ) 2 + + + 1 2 - - - x 1 x 2 D 2 D D F x ( ) x x * = x 1 x 1 * – ( ) x 2 x 2 * – ( ) … + + Matrix Form Gradient Hessian Directional Derivatives First derivative (slope) of F(x) along xi axis: Second derivative (curvature) of F(x) along xi axis: (ith element of gradient) (i,i element of Hessian) First derivative (slope) of F(x) along vector p: Second derivative (curvature) of F(x) along vector p: Example Plots x1 x1 x2 x2 1.4 1.3 0.5 0.0 1.0 Directional Derivatives Minima The point x* is a strong minimum of F(x) if a scalar d 0 exists, such that F(x*) F(x*?+?Dx) for all Dx such that d ||Dx|| 0. Strong Minimum The point x* is a unique global minimum of F(x) if F(x*)??F(x*?+?Dx) for all Dx≠0. Global Minimum The point x* is a weak minimum of F(x) if it is not a strong minimum, and a scalar d 0 exists, such that F(x*) ≤ F(x* + Dx) for all Dx such that d ||Dx|| 0. Weak Minimum Scalar Example Strong Minimum Strong Maximum Global Minimum Vector Example First-Order Optimality Condition For small Dx: If x* is a minimum, this implies: If then But this would imply that x* is not a minimum. Therefore Since this must be true for every Dx, Second-Order Condition A strong minimum will exist at x* if for any Dx≠ 0. Therefore the Hessian matrix must be positive definite. A matrix A is positive definite if: A necessary condition is that the Hessian matrix be positive semidefinite. A matrix A is positive semidefinite if: If the first-order condition is satisfied (zero gradient), then for any z ° 0. for any z. This is a sufficient condition for optimality. Example F x ( ) x 1 2 2 x 1 x 2 2 x 2 2 x 1 + + + = (Not a function of x