lecture 2 – theoretical underpinnings of mapreduce - ubc ece.pptVIP

Key idea 3:Scale out, not up! For data-intensive workloads, a large number of commodity servers is preferred over a small number of high-end servers cost of super-computers is not linear Some numbers Processing data is quick, I/O is very slow: 1 HDD = 75 MB/sec; 1000 HDDs = 75 GB/sec Data volume processed: 80 PB/day at Google; 60TB/day at Facebook (~2012) Key idea 4“Shared-nothing” infrastructure(both hard- and soft-ware) Sharing vs. Shared nothing: Sharing: manage a common/global state Shared nothing: independent entities, no common state Functional programming as key enabler No si