Lecture8Least-normsolutionsofundeterminedequations.PDFVIP

EE263 Autumn 2007-08 Stephen Boyd Lecture 8 Least-norm solutions of undetermined equations • least-norm solution of underdetermined equations • minimum norm solutions via QR factorization • derivation via Lagrange multipliers • relation to regularized least-squares • general norm minimization with equality constraints 8–1 Underdetermined linear equations we consider y = Ax where A ∈ Rm ×n is fat (m n), i.e., • there are more variables than equations • x is underspecified, i.e., many choices of x lead to the same y we’ll assume that A is full rank (m), so for each y ∈ Rm , there is a solution set of all solutions has form { x | Ax = y } = { xp + z | z ∈ N (A) } where xp is any (‘particular’) solution, i.e., Axp = y Least-norm solutions of undetermined equations 8–2 • z characterizes available choices in solution • solution has dimN (A) = n − m ‘degrees of freedom’ • can choose z to satisfy other specs or optimize among solutions Least-norm solutions of undetermined equations 8–3 Least-norm solution one particular solution is xln = AT (AAT )−1y (AAT is invertible since A full rank) in fact, xln is the solution of y = Ax that minimizes x i.e., xln is solution of optimization problem minimize x subject to Ax = y n (with variable x ∈ R ) Least-norm solutions of undetermined equations 8–4 suppose Ax = y , so A(x −