信息论编码答案1.pdfVIP

Information and Coding Theory. Detailed solutions to problems Solutions to Problems of Chapter 1 1.1) a) In order to find the entropy we need to first calculate the self informations I = log (1 / P ) : i 2 i I = log (10./4 ) =1.32 A 2 I = log (1 0/ .2 ) =2.32 B 2 I log (10./1) =3.32 C 2 I = I = log (1 0/ .05 ) =4.32 E F 2 where self informations are measured in bits. Then the entropy of the source is the mean value of these self informations: F H(X ) P I 0.4x1.32 0.2 2x.32 0.1 3x.32 20x.05 4x.32 2.22 bits / symbol = = + + + = ∑ i i i= A b) The maximum possible source entropy is: 1 H (X ) = log M = log (M) =2.58 bits / symbol max M ∑ 2 2 and the efficiency is: eff =2.22 2/.58 = 86% The information emitted by this source can be compressed 1.2) Entropy for this case is calculated as usual as: E H(X ) = P I =(1 / 2 ) log (2) +(1 / 4) log 4() +(1 / 8) log (8 ) +x2(1 / 16) log (16 ) =1. 875 bits / symbol ∑ i i 2 2 2 2 i= A the information contained by the sequence DADED is: I = I +3I + I = log (2 ) + 3 log (16 ) + log (16 ) = 17 bits seq A D E 2 2 2 1.3) The source entropy is calculated as: = + = H(X ) 0.2 x log (1 0/ .2 ) 0.8 x log (1 0/ .8 ) 0. 7219 bits / symbol