External Sorting - Worcester Polytechnic Institute (WPI)外部排序-伍斯特理工学院（WPI）.pptVIP

External Sorting Chapter 13 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing gpa order Sorting is first step in bulk loading B+ tree index. Sorting useful for eliminating duplicate copies in a collection of records Sort-merge join algorithm involves sorting. Problem: sort 1Gb of data with 1Mb of RAM. why not virtual memory? Using secondary storage effectively General Wisdom : I/O costs dominate Design algorithms to reduce I/O 2-Way Sort: Requires 3 Buffers Phase 1: PREPARE. Read a page, sort it, write it. only one buffer page is used Phase 2, 3, …, etc.: MERGE: three buffer pages used. Two-Way External Merge Sort Idea: Divide and conquer: sort subfiles and merge into larger sorts Two-Way External Merge Sort Costs for pass : all pages # of passes : height of tree Total cost : product of above Two-Way External Merge Sort Each pass we read + write each page in file. N pages in file = 2N Number of passes So total cost is: External Merge Sort What if we had more buffer pages? How do we utilize them wisely ? Phase 1 : Prepare Phase 2 : Merge Compose as many sorted sublists into one long sorted list. General External Merge Sort To sort a file with N pages using B buffer pages: Pass 0: use B buffer pages. Produce sorted runs of B pages each. Pass 1, 2, …, etc.: merge B-1 runs. Cost of External Merge Sort Number of passes: Cost = 2N * (# of passes) Example Buffer : with 5 buffer pages File to sort : 108 pages Pass 0: Size of each run? Number of runs? Pass 1: Size of each run? Number of runs? Pass 2: ??? Example Buffer : with 5 buffer pages File to sort : 108 pages Pass 0: = 22 sorted runs of 5 pages each (last run is only 3 pages) Pass 1: = 6 sorted runs of 20 pages each (last run is only 8 pages) Pass 2: 2 sorted runs, 80 pages and 28 pages Pa