Analysis of Performance Variation Using Query Expansion.pdfVIP

Analysis of Performance Variation Using Query Expansion Nega Alemayehu National Institute of Standards and Technology, 100 Bureau Drive, Stop 8940, Githersburg, MD 20899. E-mail: nega.alemayehu@ Information retrieval performance evaluation is com- figures, such as precision and recall. Although such methods monly made based on the classical recall and precision may show the improvements, such observed differences are based figures or graphs. However, important informa- not always significant. Moreover, important information tion indicating causes for variation may remain hidden under the average recall and precision figures. Identify- that can be of help in the design process may remain hidden ing significant causes for variation can help researchers beyond the average figures. Little has been done in the and developers to focus on opportunities for improve- detailed analysis of performance data and the search for ment that underlay the averages. This article presents a causes of variation compared to designing new expansion case study showing the potential of a statistical re- techniques. This is a common problem in information re- peated measures analysis of variance for testing the significance of factors in retrieval performance variation. trieval (IR) research, as indicated by Hull (1996). The TREC-9 Query Track performance data is used as a The application of statistical techniques to retrieval per- case study and the factors studied are retrieval method, formance data can provide information that is hidden be- topic, and their interaction. The result