Unsupervised Learning of the Morphology of a Natural Language：（无监督学习形态的自然语言）.pdfVIP

Unsupervised Learning of the Morphology of a Natural Language John Goldsmith University of Chicago This study reports the results of using minimum description length (MDL) analysis to model unsupervised learning of the morphological segmentation of European languages, using corpora ranging in size fro m 5,000 words to 500,000 words. We develop a set of heuristics that rapidly develop a probabilistic morphological grammar, and use MDL as our primary tool to determine whether the modi cations proposed by the heuristics will be adopted or not. The resulting gramma r matches well the analysis that would be developed by a human morphologist. In the nal section, we discuss the relationship of this style of MDL grammatica l analysis to the notion of evaluation metric in early generative grammar. 1. Introduction This is a report on the present results of a study on unsupervised acquisition of morphology.1 The central task of morphological analysis is the segmentation of words into the components that form the word by the operation of concatenation. While that view is not free of controversy, it remains the traditional conception of morphology, and the one that we shall employ here.2 Issues of interface with phonol- ogy, traditionally known as morphophonology, and with syntax are not directly addressed.3 While some of the discussion is relevant to the unrestricted set of languages, some of the assumptions made in the implementation restrict the use- ful application of the algorithms to languages in which the average number of afxes per word is less than what is found in such languages as Finnish, Hungarian, and Swahili, and we restrict our testing in the present report to more widely studied Eu- ropean languages . Our general goal, however, is the tre