第4讲-DW-Operations.pptVIP

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * HD-ITR * Space cost In this case we use what we call “chunk-offset compression.” In chunk-offset compression, for each valid array entry, we store a pair, (offsetInChunk, data). 4.2 MOLAP-Based cube * HD-ITR * Computing the cube on the array Basic cubing algorithm Multi-way array algorithm 4.2 MOLAP-Based cube * HD-ITR * Uses minimum memory Computes each aggregate in a separate pass. So no overlap in computations. 4.2.1 Basic Array Cubing Algorithm * HD-ITR * Calculate aggregation on AB Plane size is |A|*|B| Must fit AB plane into memory Better Plan? Can we take advantage of the chunking? 4.2.1 Basic Array Cubing Algorithm * HD-ITR * Better plan Scan on a chunk by chunk basis Ac is chunk size for A and Bc for B Note: Ac not necessarily equal to Bc A B C 2 1 Scan all upper left chunks, store result Keep track of running total only Compute next chunks (one to right) Patch results together Only need memory for |Ac|*|Bc| instead of |A|*|B| 4.2.1 Basic Array Cubing Algorithm * HD-ITR * How to calculate entire cube? Array requires computing all aggregates: BC, AC, A, B, and C also Simple approach: Scan ABC to compute each one Can we do better? We can do better! Only need to scan AB to compute A, B More efficient Quicker Only need BC for C 4.2.1 Basic Array Cubing Algorithm * HD-ITR * Example: To compute cube for ABC Scan ABC for AB Rescan ABC for BC, again for AC Scan AB for A, rescan for B Scan BC for C (could also be used to find B) 6 scans 4.2.1 Basic Array Cubing Algorithm * HD-ITR * Computing the cube on the array Basic cubing algorithm Multi-way array algorithm 4.2 MOLAP-Based cube * HD-ITR * 4.2.2 Multi-way array algorithm This algorithm overlaps the computations of the different groupbys, thus avoiding the multiple scans required by the na?ve algorithm. Two possibilities: Sufficient memory to compute all groupbys in one scan Not very realistic In