Numerical Linear Algebra in the Streaming Model IBM数值线性代数中的流模型 IBM.pptVIP

Numerical Linear Algebra in the Streaming Model Ken Clarkson - IBM David Woodruff - IBM The Problems Given n x d matrix A and n x d’ matrix B, we want estimators for The matrix product AT B The matrix X* minimizing ||AX-B|| A slightly generalized version of least-squares regression Given integer k, the matrix Ak of rank k minimizing ||A-Ak|| We consider the Frobenius matrix norm: root of sum of squares General Properties of Our Algorithms 1 pass over matrix entries, given in any order (allow multiple updates) Maintain compressed versions or “sketches” of matrices Do small work per entry to maintain the sketches Output result using the sketches Randomized approximation algorithms Matrix Compression Methods In a line of similar efforts… Element-wise sampling [AM01], [AHK06] Sketching / Random Projection: maintain a small number of random linear combinations of rows or columns [S06] Row / column sampling [DK01], [DKM04], [DMM08] Usually more than 1 pass Here: sketching Outline Matrix Product Regression Low-rank approximation Approximate Matrix Product A and B have n rows, we want to estimate ATB Let S be an n x m sign (Rademacher) matrix Each entry is +1 or -1 with probability ? m is small, to be specified Entries are O(log 1/δ)-wise independent Sketches are STA and STB Our estimate of ATB is [ATS][STB]/m Easy to maintain sketches given updates O(m) time per update, O(mc log(nc)) bits of space for STA and STB Output [ATS][STB]/m using fast rectangular matrix multiplication Expected Error and a Tail Estimate Using linearity of expectation, E[ATSSTB/m] = ATE[SST]B/m = ATB Moreover, for δ, ε 0, there is m = O(log 1/ δ) ε-2 so that Pr[||ATSSTB/m-ATB|| ε ||A|| ||B||] · δ (again ||C|| = [Σi, j Ci, j2]1/2) This tail estimate seems to be new Follows from bounding O(log 1/δ)-th moment of ||ATSSTB/m-ATB|| Improves space of Sarlós’ 1-pass algorithm by a log c factor Matrix Product Lower Bound Our algorithm is space-optimal for constant δ a new lower bound