lecnote_5.pptVIP

第五讲 并行算法设计基础：PCAM 关于并行程序的性能 Speedup Efficiency Amdahl’s Law 回答一个问题：值得并行吗？ Confronting Race Conditions Race condition Mutual exclusion Critical region Lock 子任务的协调: 同步/数据交换 性能优化 并行算法的判定：scalability 使用的处理器执行内核数量越多，是否一定能够提高问题求解的效率？ Idea Behind Amdahl’s Law Derivation of Amdahl’s Law Speedup is ratio of execution time on 1 processor to execution time on p processors Execution time on 1 processor is f + 1-f Execution time on p processors is at least f + 1-f /p Amdahl’s Law Is Too Optimistic Amdahl’s Law ignores parallel processing overhead Examples of this overhead include time spent creating and terminating threads Parallel processing overhead is usually an increasing function of the number of processors Other Optimistic Assumptions Amdahl’s Law assumes that the computation divides evenly among the processors In reality, the amount of work does not divide evenly among the processors Processor waiting time is another form of overhead Graph with Workload Imbalance Added More General Speedup Formula ? n,p Speedup for problem of size n on p CPUs ? n Time spent in sequential portion of code for problem of size n ? n Time spent in parallelizable portion of code for problem of size n ? n,p Parallel overhead Amdahl’s Law: Maximum Speedup The Amdahl Effect Illustration of the Amdahl Effect Using Amdahl’s Law Program executes in 5 seconds Profile reveals 80% of time spent in function alpha, which we can execute in parallel What would be maximum speedup on 2 processors? New execution time ≥ 5 sec / 1.67 3 seconds Amdahl定律告诉我们什么有意义的事情？ 可并行计算量占主要比重 通过并行化，增加的额外计算量所占比重有限：这些计算量由各个处理器/执行内核分摊 要有足够的并行度 并行子任务数量 ，实现负载均衡：问题规模越大，并行度也越大 数据并行 大规模数据处理 数据规模与处理器/执行内核的规模比率要适当 只有上述三个特征都满足了，采用并行处理技术才有意义 数据并行与任务并行 数据并行：在一个数据集合的不同元素上，分别执行相同的操作 每个操作涉及其中的一个子集 包括一个、多个元素 各个子集包含的元素数量相同 子集的执行顺序对最终结果没有影响 情形1: 子集之间完全没有关系 情形2:不同子集的结果要规约到同一个数据上 数据集合越大，这样的子集数量也越大 例如：线性方程组求解、N-Body问题、求素数问题等 任务并行：从问题域的结构特征出发，采用功能-子功能分解的办法，寻找可以并发执行的子任务 子任务是概念上的划分，与问题的规模无关 子任务的数量非常有限 总有子任务内存在数据并行 例如：全球气候模拟包括大气模拟、陆