8.Linearleast-squares.PDFVIP

EE103 (Fall 2011-12) 8. Linear least-squares • definition • examples and applications • solution of a least-squares problem, normal equations 8-1 Definition overdetermined linear equations Ax = b (A is m × n with m n) if b ∈ range(A), cannot solve for x least-squares formulation m n 1/2 minimize Ax − b = (a x − b )2 ij j i i=1 j =1 • r = Ax − b is called the residual or error • x with smallest residual norm r is called the least-squares solution 2 • equivalent to minimizing Ax − b Linear least-squares 8-2 Example  2 0   1      A = −1 1 , b = 0 0 2 −1 least-squares solution minimize (2x − 1)2 + (−x + x )2 + (2x + 1)2 1 1 2 2 to find optimal x , x , set derivatives w.r.t. x and x equal to zero: 1 2 1 2 10x1 − 2x2 − 4 = 0, −2x1 + 10x2 + 4 = 0 solution x1 = 1/3, x2 = −1/3 (much more on practical algorithms for LS problems later) Linear least-squares 8-3 r2