在线联机业务分析平台.pdf

Towards Building Interactive and Online Analytics Systems Feifei Li /~lifeifei/ University of Utah Interactive and Online Data Analytics Systems  Interactive query and analytics: – Issue Queries as You Wish  Online query and analytics: – Control the tradeoff between result quality and query/analytics efficiency  Rich analytical support: – Support query and analytical operations for knowledge discovery through easy-to-use and intuitive query abstraction and interfaces 3 Interactive and Online Data Analytics Systems  Geo-tagged tweets as an example (crawling since May 2014) – 1.4 geo-tagged billion tweets so far – 1.8 TB – 3-4 million new tweets per day  Interactive and online analytics is a must-have: – Doing ETL and building a data warehouse is expensive and restrictive 4 System Interface 5 Interactive and Online Data Analytics Systems 6 Another example: The MesoWest Project 7 Interactive and Online Data Analytics Systems  Key challenges and opportunities – Interactive: In-Memory Cluster Based Computation – Online: Accuracy vs. Efficiency Tradeoff: existing systems are binary, either no results or wait for unknown amount of time – Learning: Real-time Tracking, Monitoring and Prediction: analyzing incoming data in conjunction with historical data (using machine-learning based, data driven approach) 8 100 TB on 1000 machines ½ - 1 Hour 1 - 5 Minutes Hard Disks Memory In memory computation over a cluster Rich types of queries and analytics: spatial/multimedia data SELECT * SELECT * FROM points FROM points SORT BY (x - 2)*(x - 2) WHERE POINT(x, y