Efficient Processing of Extra-grammatical Sentences Comparing and Combining two approaches.pdf

Efficient Processing of Extra-grammatical Sentences: Comparing and Combining two approaches to Robust Stochastic parsing Marita Ailomaa2, Vladim??r Kadlec1, Jean-Ce?dric Chappelier2, and Martin Rajman2 1 Faculty of Informatics, Masaryk University Botanicka? 68a, 602 00 Brno, Czech Republic E-mail: xkadlec@fi.muni.cz 2 Artificial Intelligence Laboratory, Computer Science Department Swiss Federal Institute of Technology (EPFL) 1015 Lausanne, Switzerland E-mail: {marita.ailomaa,jean-cedric.chappelier,martin.rajman}@epfl.ch Abstract. This paper compares two techniques for robust parsing of extra-gram- matical natural language that might be of interest in large scale Textual Data Analysis applications. The first one returns a “correct” derivation for any extra- grammatical sentence by generating the finest corresponding most probable optimal maximum coverage. The second one extends the initial grammar by adding relaxed grammar rules in a controlled manner. Both techniques use a stochastic parser that selects a “best” solution among multiple analyses. The techniques were tested on the ATIS and Susanne corpora and experimental results, as well as conclusions on performance comparison, are provided. Keywords: Robust, Parsing, Coverage. 1 Introduction Formal grammars are traditionally used in NLP applications to describe well- formed sentences. But in large scale Textual Analysis applications it is not practical to rely exclusively on a formal grammar because of the large frac- tion of sentences that will receive no analysis. This undergeneration problem has lead to a whole field of research called robust parsing, where the goal is to find domain-independent, efficient parsing techniques that return a cor- rect or usefully “close” analysis for almost all of the input sentences [Carroll and Briscoe, 1996]. Such techniques need to handle not only the problems of undergeneration but also the increased ambiguity which is usually a conse- quence of the robustification of the pars