The Web as a Graph, Models and Algorithms.ppt

The Web as a Graph, Models and Algorithms Sridhar Rajagopalan IBM Almaden Research Center Graphs in Computer Science: How do they help? What do they model? An abstraction in Core CS. Examples: VLSI Circuits, Communication Networks, Logical flow of a computer program, Data structures. An abstraction for data and relationships. Examples: The Web, Social Networks, Flows and Flow Networks, Biological Data, Taxonomies, Citations, Explicit relations within a DB system. What aspects are studied? Algorithms, Data Structures and Complexity Theory. Characterization and Modeling of Graphs. Implementations of Graph Algorithms in Specific contexts. What is new? Some graphs are getting very very large. Several Web crawls have over 2 Billion pages (nodes) and 10 times as many edges. Some social networks (Telephone call graphs, for instance) are huge. Sensors and small devices will output enormous amounts of data. Why and how does scale change the game? It is not about an instance, it is about THE instance. Systems and Software is designed from scratch to solve one (or a few) instances of the problem. Properties of the instance are important. Generality and genericity of the code base are not critical. An answer to a single instance can be worth a lot. A new kind of algorithm and system design which is very engineering orientated. Non-viability of the random access model. Hardware and software co-design.. “Time to market” considerations encourage careful approximations. Statistical approaches are valuable. Do not need the complete answer, even a partial high quality result can be very valuable. What does a system and programming model for processing large data sets have to do? System and programming model for processing large graphs. Exceptions will occur. Components (both hard and soft) will fail. Data structures will exceed available memory. Aware of statistical issues. Approximate or incomplete results are usually good enough. What happens when you string (even well underst