Sofarwehavecovered….pptVIP

So far we have covered … Basic visualization algorithms Parallel polygon rendering Occlusion culling Tera-scale Data Visualization Big Data? Big data collection vs. big data objects Big data collection: aggregates of many data sets (multi-source, multi-disciplinary, hetreogeneous, and maybe distributed) Big data objects: single object too large For main memory For local disk Even for remote disk Big Data Objects As a result of large-scale simulations (CFD, weather modeling, structural analysis, etc) A sample of problems Data management – data models, data structures, storage, hierarchy etc. Too big for local memory (e.g. 10GB time-varying data) Too big for local disk (e.g. 650GB simulation data) High bandwidth and latency Possible Solutions Write-A-Check Approach Possible Solutions (2) Write-A-Check Approach 2: Possible Solutions (3) Perhaps a better approach … Data reduction techniques Goal: Reduce the memory/disk/network resource requirements Memory Hierarchy Indexing Compression Multiresolution Data mining and feature extractions Memory Hierarchy A system approaches Break the data in pieces Retrieve only the relevant pieces Demand-driven Sparse traversal using index Break Data in Pieces Although O.S supports this long time ago… (VM) Break Data in Pieces (2) It is better for the application to decide how the data should be subdivided Caution: Don’t be too algorithm specific You don’t want to have one file layout for each viz algorithm Most of the vis algorithms have similar memory access patterns Issues: fast addressing without bloat the data Demand-Driven Data Fetch Virtual Memory typically won’t do a good job - do not know what are the necessary blocks An application control data retrieval system is needed Fast block retrieval Overlap I/O with Computation Smart pre-fetch … Sparse Traversal Memory hierarchy approach works the best when the underlying algorithms only need a sparse traversal of the data with hig