Module 2 - Student Background Knowledge Google云计算系列课程知识讲稿.ppt

? Spinnaker Labs, Inc. Ports Within an IP address, a port is a sub-address identifying a listening program Allows multiple clients to connect to a server at once Example: Web Server (1/3) The server creates a listener socket attached to a specific port. 80 is the agreed-upon port number for web traffic. Google Cluster Computing Faculty Training Workshop Module II: Student Background Knowledge This presentation includes course content ? University of Washington Redistributed under the Creative Commons Attribution 3.0 license. All other contents: Background Topics Programming Languages Systems: Operating Systems File Systems Networking Databases Programming Languages MapReduce is based on functional programming map and fold FP is taught in one quarter, but not reinforced “Crash course” necessary Worksheets to pose short problems in terms of map and fold Immutable data a key concept Multithreaded programming Taught in OS course at Washington Not a prerequisite! Students need to understand multiple copies of same method running in parallel File Systems Necessary to understand GFS Comparison to NFS, other distributed file systems relevant Databases Concept of shared consistency model Consensus ACID characteristics Journaling Multi-phase commit processes Parallelization Synchronization Parallelization Idea Parallelization is “easy” if processing can be cleanly split into n units: Parallelization Idea (2) In a parallel computation, we would like to have as many threads as we have processors. e.g., a four-processor computer would be able to run four threads at the same time. Parallelization Idea (3) Parallelization Idea (4) Parallelization Pitfalls But this model is too simple! How do we assign work units to worker threads? What if we have more work units than threads? How do we aggregate the results at the end? How do we know all the workers have finished? What if the work cannot be divided into completely separate tasks? What is the common theme of all of these p