专题论坛大数据课件.pptVIP

The Arecibo Telescope World’s largest radio telescope Diameter : 305 m (1,000 ft) Area : 18 acres Location: Arecibo, Puerto Rico The P-ALFA surveys 800 Terabytes in 5 years Software Driven Telescope from few, large, expensive, directional dishes to many, small, cheap, omni directional antennae a large number of high-speed input streams (2Gbps per antenna, 25,000 antennae in an area of 340 km in diameter) Data size Challenge 1: It’s the data, stupid! Data complexity Key/value store Column store Document store Graph Systems Big data drives tomorrow’s economy. ? The value of big data lies in its degree of connectedness. ? Existing systems cannot handle rich connectedness of big data. RDBMS and Rich Relationships ? Performance of multi-way joins is very poor in RDBMS ? Managing data of rich connectedness requires multi-way Joins in RDBMS Trinity ? A general purpose, distributed, in memory graph system ? Online graph query processing ? Offline graph analytics Trinity Performance Highlight ? Online query processing : – visiting 2.2 million users (3 hop neighborhood) on Facebook: = 100ms – foundation for graph-based service, e.g., entity search ? Offline graph analytics : – one iteration on a 1 billion node graph: = 60sec – foundation for analytics, e.g., social analytics People Search Demo Multi-way Join vs. Graph Traversal Company Incident Problem … ID Company ID1 ID2 ID … Incident ID3 ID4 ID … Problem RDBMS Trinity Challenge 2: Interpretation of Big Data ? IBM Watson: – Runs on 2,880 cores, 15 terabytes of RAM, and 80kW of power ? A human brain: – Runs on a tuna fish sandwich and a glass of water answering the question unconstrained natural language inferencing reasoning domain specific language simple calculation Human (Turing Test) SIRI Watson Wolfram Alpha Google/Bing ? the Eternal Quest understanding the question SQL calculator Turning the Web into a Database What you see when you look