English for IT and EE-04.pptVIP

* 9 Quantities of information Information theory is based on probability theory and statistics. The most important quantities of information are entropy, the information in a random variable, and mutual information, the amount of information in common between two random variables. 最重要的信息量是熵即随机变量中的信息，以及互信息即两个随机变量之间的共有信息量。 * 9，10 The former quantity indicates how easily message data can be compressed while the latter can be used to find the communication rate across a channel.6 The choice of logarithmic base determines the unit of information entropy that is used. The most common unit of information is the bit, based on the binary logarithm. 最常见的信息单位是基于以2为底对数的比特。 前者指出消息数据压缩的难易程度，后者确定信道的通信速率。 * 11 Coding theory Coding theory is the most important and direct application of information theory. It can be subdivided into source coding theory and channel coding theory. Using a statistical description for data, information theory quantifies the number of bits needed to describe the data, which is the information entropy of the source. Data compression (source coding). There are two formulations for the compression problem: 通过采用数据的统计描述，信息论确定了描述数据所需要的比特数即信源的信息熵。 * 12 Lossless data compression — the data must be reconstructed exactly; Lossy data compression: allocates bits needed to reconstruct the data, within a specified fidelity level measured by a distortion function. This subset of Information theory is called rate-distortion theory.7 有损数据压缩是在给定保真度条件下分配重建数据所需比特数，保真度由失真函数来度量。信息论的这个分支称为率－失真理论。 * 13 Error-correcting codes (channel coding). While data compression removes as much redundancy as possible, an error correcting code adds just the right kind of redundancy (i.e. error correction) needed to transmit the data efficiently and faithfully across a noisy channel. 数据压缩是尽可能地除去冗余度，而纠错编码则是增加适当的冗余度，使数据在含噪信道中更有效更准确地传输。 * 14 This division of coding theory into compression and transmission is justified by the information transmission theorems, or source-channel separatio