Frequency Value Grammar and Information Theory.pdfVIP

1 Frequency Value Grammar and Information Theory Mar.18th, 2004 Copyright 2004 Asa M. Stepak Abstract: This paper convincingly shows that previous efforts to calculate the Entropy of written English are based upon inadequate n-gram models that will result in an overestimation of the Entropy. Frequency Value Grammar, FVG, is not based upon corpus based word frequencies such as in Probability Syntax, PS. The characteristic ?frequency values? in FVG are based upon ?information source? probabilities which are implied by syntactic category and constituent corpus frequency parameters. The theoretical framework of FVG has broad applications beyond that of formal syntax and NLP. In this paper, I demonstrate how FVG can be used as a model for improving the upper bound Entropy calculation of written English. Generally speaking, when a function word precedes an open class word within a phrasal unit, the backward bi-gram analysis will be homomorphic with the ?information source? probabilities and will result in word frequency values more representative of cognitive object co-occurrences in the ?information source?. 1. Introduction I previously laid the groundwork for Frequency Value Grammar(FVG) in papers I submitted in the proceedings of the 4th International Conference on Cognitive Science (2003), Sydney Australia, and Corpus Linguistics Conference (2003), Lancaster, UK. FVG is a formal syntax theoretically based in large part on Information Theory principles. FVG relies on dynamic physical principles external to the corpus which shape and mould the corpus whereas generative grammar and other formal syntactic theories are based exclusively on patterns (fractals) found occurring within the well-formed portion of the corpus. However, FVG should not be confused with Probability Syntax, (PS), as described by Manning (2003). PS is a corpus based approach that will yield the probability distribution of possible syntax construc