象征性的布尔操作的有序二元决策_2.pptVIP

Data Intensive Super Computing Motivation 200+ processors 200+ terabyte database 1010 total clock cycles 0.1 second response time 5￠ average advertising revenue Google’s Computing Infrastructure System ~ 3 million processors in clusters of ~2000 processors each Commodity parts x86 processors, IDE disks, Ethernet communications Gain reliability through redundancy software management Partitioned workload Data: Web pages, indices distributed across processors Function: crawling, index generation, index search, document retrieval, Ad placement A Data-Intensive Super Computer (DISC) Large-scale computer centered around data Collecting, maintaining, indexing, computing Similar systems at Microsoft Yahoo Google’s Economics Making Money from Search $5B search advertising revenue in 2006 Est. 100 B search queries ? 5￠ / query average revenue Google’s Programming Model MapReduce Map computation across many objects E.g., 1010 Internet web pages Aggregate results in many different ways System deals with issues of resource allocation reliability DISC: Beyond Web Search Data-Intensive Application Domains Rely on large, ever-changing data sets Collecting maintaining data is major effort Many possibilities Computational Requirements From simple queries to large-scale analyses Require parallel processing Want to program at abstract level Hypothesis Can apply DISC to many other application domains The Power of Data + Computation 2005 NIST Machine Translation Competition Translate 100 news articles from Arabic to English Google’s Entry First-time entry Highly qualified researchers No one on research team knew Arabic Purely statistical approach Create most likely translations of words and phrases Combine into most likely sentences Trained using United Nations documents 200 million words of high quality translated text 1 trillion words of monolingual text in target language During competition, ran on 1000-processor cluster One hour per sentence (gotten faster now) 200