1 Text analysis meets corpus linguistics.pdf

1 Text analysis meets corpus linguistics Hannah Kermes, Stefan Evert Institut für Maschinelle Sprachverarbeitung, Azenbergstr. 12, 70174 Stuttgart, Germany {kermes,evert}@ims.uni-stuttgart.de 1 Introduction In recent years, there has been rising interest to using evidence derived from automatic syntactic analysis in large-scale corpus studies. Ideally, of course, corpus linguists would prefer to have access to the wealth of structural and featural information provided by a full parser based on a complex grammar formalism. However, to date such parsers achieve neither the speed nor the robustness needed to process hundreds of millions of words. Beside this practical limitation, there are at least two more fundamental problems with this approach. Firstly, complex grammars tend to produce highly ambiguous output. Without extensive lexical and semantic knowledge, there will often be thousands of different analyses for any given sentence. Secondly, full parser usually embrace a particular theoretical perspective, embodied in the grammar formalism they use. If the researcher’s perspective on syntax is different from that of the parser, he or she will find it difficult, if not outright impossible, to apply the parser’s analyses to the research question. 1 Chunk parsers, on the other hand, were expressly designed for the robust processing of large amounts of text, including sentence fragments. Robustness is achieved mostly by avoiding problematic decisions, such as where to attach prepositional phrases. As a consequence, the local structures built by chunk parsers are largely independent from any particular syntactic theory. However, they also provide much less evidence for the syntactic structure of sentences and for other linguistic phenomena. In this paper, we want to show that a chunk parser with an extended chunk definition can provide analyses of sufficient depth and enriched with morpho-syntactic as well as lexical-semantic information