浅谈信息理论对数字通信系统发展的指导.ppt

浅谈信息理论对数字通信系统发展的指导 马 啸 中山大学信息科学与技术学院 Email:maxiao@ 主要内容 Shannon信息论 信源编码 信道编码 深空通信 Modem 无线移动通信 Shannon其人 /wiki/Claude_Shannon Claude Elwood Shannon (April 30, 1916 – February 24, 2001), an American electrical engineer and mathematician, the father of information theory. Shannon is famous for having founded information theory with one landmark paper published in 1948. But he is also credited with founding both digital computer and digital circuit design theory in 1937, when, as a 21-year-old masters student at MIT, he wrote a thesis demonstrating that electrical application of Boolean algebra could construct and resolve any logical, numerical relationship. It has been claimed that this was the most important masters thesis of all time. Mrs. Shannon with the statue. (photograph courtesy of J. OSullivan) 通信的数学理论 Claude E. Shannon， A Mathematical Theory of Communication, Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, 1948. 数字通信系统模型 信息理论基本概念 两个离散随机变量 X和Y 熵的意义 随机变量的不确定性，含糊度，随机变量所含的信息量 条件熵 掷均匀硬币：1 bit 猜数字（Yes-No） 称球问题 信息量的无语义性 信息领域其他有趣的问题（不均匀硬币） 掷硬币 信源 信源是产生消息的源，形式多样 根据信息源输出信号的性质不同，可分为模拟信源和离散信源 模拟信源输出连续幅度的信号，如语音、图像、视频等 离散信源输出离散的符号序列或文字，如文本文字等 模拟信源可以通过抽样和量化变换为离散信源 离散信源又可以通过信源编码化为二进制 Every thing is binary! 信源编译码 把信源发出的信息转化为二进制序列 用最简洁的形式表达出来，消除冗余信息 极限是信源的熵（模糊度） 编码实例 例假设信源 Huffman编码实例 实质：是一个贪心算法；大概率符号用短码表示 Huffman编码的教学意义 In 1951, David A. Huffman and his MIT information theory classmates were given the choice of a term paper or a final exam. The professor, Robert M. Fano, assigned a term paper on the problem of finding the most efficient binary code. Huffman, unable to prove any codes were the most efficient, was about to give up and start studying for the final when he hit upon the idea of using a frequency-sorted binary tree and quickly proved this method the most efficient. In doing so, the student outdid his professor, who had worked with information theory inventor Claude Shannon to develop a similar code. Huffman avoided the major flaw of the s