Klex A Finite-State Transducer Lexicon of Korean.pdf

Klex: A Finite-State Transducer Lexicon of Korean Na-Rae Han Department of Linguistics, University of Pennsylvania, Philadelphia, PA 19104, USA, nrh@, WWW home page: /～nrh/klex.html Abstract. This paper describes the implementation and system details of Klex, a finite-state transducer lexicon for the Korean language, devel- oped using XRCE’s Xerox Finite State Tool (XFST). Klex is essentially a transducer network representing the lexicon of the Korean language with the lexical string on the upper side and the inflected surface string on the lower side. Two major applications for Klex are morphological analysis and generation: given a well-formed inflected lower string, a language- independent algorithm derives the upper lexical string from the network and vice versa. Klex was written to conform to the part-of-speech tagging standards of the Korean Treebank Project, and is currently operating as the morphological analysis engine for the project. 1 Introduction Korean is a highly agglutinative language, with productive use of post-position markers following nouns, verbal ending suffixes for verbs and adjectives, as well as frequent use of derivational morphology. Breaking up Korean words into smaller morphological components is the necessary first step in any natural lan- guage processing system of Korean, as well as an essential task for computerized language-learning tools. Klex is a fully operational Korean lexicon system whose underlying mathematical representation is a form of data structure known as the finite-state transducer or FST.1 The transducer’s upper level consists of strings representing a sequence of lexical forms of morphemes, each followed by their part-of-speech tag; the lower level consists of the fully inflected surface forms produced by concatenation of the morphemes, with relevant phonological and morphotactic processes applied: 1 The name Klex is used somewhat ambiguously: it refers to the binary FST network that constitutes the lexicon; it als