demmel@csberkeleyedu.ppt

2/4/2004 CS267 Lecure 5 CS267 Lecture 2 Basic Shared Memory Architecture Processors all connected to a large shared memory Where are caches? Outline Evolution of Hardware and Software CPUs getting exponentially faster than memory they share Hardware evolves to try to match speeds Program semantics evolve too Programs may change from correct to buggy, unless programmed carefully Performance evolves as well Well tuned programs today may be inefficient tomorrow Goal: teach a programming style likely to stay correct, if not always as efficient as possible Use locks to avoid race conditions Current research seeks best of both worlds Example: Sharks and Fish (part of next homework) Processor-DRAM Gap (latency) Shared Memory Code for Computing a Sums = f(A[0]) + f(A[1]) Approaches to Building Parallel Machines Shared Cache: Advantages and Disadvantages Advantages Placement of data in shared cache identical to single processor case Only one copy of any cached block Can’t have values of same memory location in different caches Fine-grain sharing is possible “Good” Interference One processor may prefetch data for another Can share data within a cache line without moving line Disadvantages Bandwidth limitation “Bad” Interference One processor may flush another processors data Limits of Shared Cache Approach Assume: 1 GHz processor w/o cache = Need up to 4 GB/s inst BW per processor (32-bit) = Need up to 1.2 GB/s data BW at 30% load-store Need 5.2 GB/s of bus bandwidth per processor! But typical off-chip bus bandwidth is closer to 1 GB/s Evolution of Shared Cache Alliant FX-8 (early 1980s) eight 68020s with x-bar to 512 KB interleaved cache Encore Sequent (1980s) first 32-bit micros (N32032) two to a board with a shared cache Disappeared for a while, and then … Cray X1 shares L3 cache IBM Power 4, Power 5, BlueGene nodes share L2 cache If switch and cache on chip, may have enough bandwidth again Approaches to Building Parallel Machines Intuitive Memory Model Reading an