recognizing text genres with simple metrics using discriminant analysis（识别文本类型使用判别分析与简单的度量标准）.pdfVIP

Recognizing Text Genres with Simple Metrics Using Discriminant Analysis Jussi Karlgren Douglass Cutting jussi@sics.se cutting@ Swedish Institute of Computer Science Apple Computer Box 1263, S – 164 28 Kista, Stockholm, Sweden Cupertino, CA 95014, USA February 5, 2008 4 9 Abstract 9 1 A simple method for categorizing texts into pre-determined text genre categories using the statistical standard technique t of discriminant analysis is demonstrated with application to the Brown corpus. Discriminant analysis makes it possible use c O a large number of parameters that may be specific for a certain corpus or information stream, and combine them into a small number of functions, with the parameters weighted on basis of how useful they are for discriminating text genres. An 0 application to information retrieval is discussed. 2 1 Text Types v 8 There are different types of text. Texts “about” the same thing may be in differing genres, of different types, and 0 of varying quality. Texts vary along several parameters, all relevant for the general information retrieval problem 0 0 of matching reader needs and texts. Given this variation, in a text retrieval context the problems are (i) identifying 1 genres, and (ii) choosing criteria to cluster texts of the same genre, with predictable precision and recall. This 4 should not be confused with the issue of identifying topics, and