Entropy as Computational Complexity熵的计算复杂度.pptVIP

Enｔｒｏｐｙ as Computational ComplexityComputer Science Replugged Tadao Takaoka Department of Computer Science University of Canterbury Christchurch, New Zealand Genera framework and motivation If the given problem is partially solved, how much more time is needed to solve the problem completely. Algorithm Input data Saved data Save Recover Output Time =Time spent + Time to be spent A possible scenario: Suppose computer is stopped by power cut and partially solved data are saved by battery. After a while power is back on , data are recovered, and computation resumes. How much more time ? Estimate from the partially solved data. Entropy Thermodynamics kloge(Q), Q number of states in the closed system, k Boltzman constant Shannon’s information theory –?[i=1, n] pilog(pi), where n=26 for English Our entropy, algorithmic entropy to describe computational complexity Definition of entropy Let X be the data set and X be decomposed like S(X)=(X1, …, Xk). Each Xi is solved. S(X) is a state of data, and abbreviated as S. Let pi=|Xi|/|X|, and |X|=n. The entropy H(S) is defined by H(S) = ?[i=1, k]|Xi|log(|X|/|Xi|) = -n?[i=1,k]pilog(pi) ? pi = 1, 0 ? H(S) ? nlog(k), maximum when all |Xi| are equal to 1/k. Amortized analysis The accounting equation at the i-th operation becomes ai = ti - ΔH(Si), 　actual time – decrease of entropy 　　where ΔH(Si) = H(Si-1) - H(Si). Let T and A be the actual total time and the amortized total time. Summing up ai for i=1, ..., N, we have A = T + H(SN) - H(S0), or T = A + H(S0) - H(SN). In many applications, A=0 and H(SN)=0, meaning T=H(S0) For some applications, ai = ti - cΔH(Si) for some constant c Problems analyzed by entropy Minimal Mergesort : merging shortest ascending runs Shortest Path Problem: Solid parts solved. Make a single shortest path spanning tree Minimum Spanning Tree : solid part solved. Make a single tree souurce Three examples with k=3 (1) Ｍｉｎｉｍａｌ mergesort Time =O(H(S)) Xi are ascending runs