scalinginnumericallinearalgebra.pptVIP

ScalinginNumerical Linear Algebra James Demmel EECS and Math Depts. CITRIS Chief Scientist UC Berkeley 20 May 2002 Outline Technology Trends and Methodology Dense Linear Algebra Sparse Direct Solvers for Ax= b Automatic Performance Tuning of Sparse Kernels Sparse Iterative Solvers for Ax=b Success Stories (with NERSC, LBNL) Scattering in a quantum system of three charged particles (Rescigno, Baertschy, Isaacs and McCurdy, Dec. 24, 1999). Tech Trends Methodology Performance depends on Maximum Flop rate Memory latencies and bandwidths Network latencies and bandwidths The Flop rate is growing faster than the other quantities Latencies improving most slowly Moving to Grid makes things worse Methodology Develop performance models Plug tech trend lines into models Predict scalability, identify bottlenecks Fix bottlenecks or find new algorithms Work in progress… Winner of TOPS 500, by year End-to-end message latencies are not improving (much) Software Overhead is a culprit ScaLAPACKA Parallel DistributedDense Linear Algebra Library ScaLAPACK Team Susan Blackford, UT Jaeyoung Choi, Soongsil U Andy Cleary, LLNL Ed DAzevedo, ORNL Jim Demmel, UCB Inder Dhillon, UCB /scalapack Jack Dongarra, UT/ORNL Sven Hammarling, NAG Greg Henry, Intel Osni Marques, NERSC Antoine Petitet, UT Ken Stanley, UCB David Walker, Cardiff U Clint Whaley, UT scalapack@ Possible Data Layouts ScaLAPACK supports all layouts 2D block cyclic recommended, for load balance and BLAS3 ScaLAPACK Structure Parallelism in ScaLAPACK Level 3 BLAS block operations All the reduction routines Pipelining QR Iteration, Triangular Solvers, classic factorizations Redundant computations Condition estimators Static work assignment Bisection Task parallelism Sign function eigenvalue computations Divide and Conquer Tridiagonal and band solvers, symmetric eigenvalue problem and Sign function Cyclic reduction Reduced system in the band solver ScaLAPACK Performance Models (1)ScaLAPAC