Opt-lec-intro.ppt

* Rates of Convergence Many of the algorithms discussed in this course do not find a solution in a finite number of steps Instead these algorithms compute a sequence of approximate solutions that we hope get closer and closer to a solution When discussing such an algorithm two questions are often asked: Does it converge? How fast does it converge? If an algorithm converges in a finite number of steps, the cost of that algorithm is often measured by counting the number of steps required counting the number of arithmetic operations required * Rates of Convergence When the number of operations/steps required to find an exact solution is infinite, some other measure of efficiency must be used The rate of convergence is one such measure that describes how quickly the estimates of the solution approach the exact solution * Rates of Convergence Let us assume that we have ideal convergence behavior * Rates of Convergence In many situations, people use a sort of shorthand and only refer to the convergence rate without mention of the rate constant For quadratic rates of convergence this is not too misleading However, in the linear case, the rate constant plays an important role (close to one, close to zero) Though it is generally true that higher rates of convergence often represent improvements in performance, this is not guaranteed An algorithm with a linear rate of convergence can sometimes be effective in a practical setting Some Remarks: * Taylor Series The approximation obtained is a polynomial. It can be applied whenever the function has derivatives and it has many uses: It is used to estimate the value of the function near the given point (when the function is difficult to evaluate directly) The derivatives and integrals of the approximation can be used to estimate the derivatives and integrals of the original function It is used to derive many algorithms for finding zeros of functions for minimizing functions, etc. * Taylor Series * Taylor Ser