shannon entropy axiomatic characterization and application香农熵公理化特性和应用.pdf

SHANNON ENTROPY: AXIOMATIC CHARACTERIZATION AND APPLICATION C. G. CHAKRABARTI AND INDRANIL CHAKRABARTY Received 18 February 2005 We have presented a new axiomatic derivation of Shannon entropy for a discrete proba- bility distribution on the basis of the postulates of additivity and concavity of the entropy function. We have then modified Shannon entropy to take account of observational un- certainty.The modified entropy reduces, in the limiting case, to the form of Shannon dif- ferential entropy. As an application, we have derived the expression for classical entropy of statistical mechanics from the quantized form of the entropy. 1. Introduction Shannon entropy is the key concept of information theory [12]. It has found wide ap- plications in different fields of science and technology [3, 4, 5, 7]. It is a characteristic of probability distribution providing a measure of uncertainty associated with the proba- bility distribution. There are different approaches to the derivation of Shannon entropy based on different postulates or axioms [1, 8]. The object of present paper is to stress the importance of the properties of additivity and concavity in the determination of functional form of Shannon entropy and its gen- eralization. The main content of the paper is divided into three sections. In Section 2, we have provided an axiomatic derivation of Shannon entropy on the basis of the prop- erties of additivity and concavity of entropy function. In Section 3, we have generalized Shannon entropy and introduced the notion of total entropy to take account of observa- tional uncertainty. The entropy of continuous distribution, called the differential entropy, has been obtained as a limiting value . In Section 4, the differential entropy along with the quantum uncertainty relation has been used to derive the expression of classical entropy in statistical mechanics. 2. Shannon entropy: axiomatic characterization Let ∆n be the set of all finite discrete