The University of Sheffield’s TREC 2004 Qamp;A Experiments.pdfVIP

The University of Sheffield’s TREC 2004 QA Experiments Robert Gaizauskas, Mark A. Greenwood, Mark Hepple, Ian Roberts, Horacio Saggion r.gaizauskas,m.greenwood, @dcs.shef.ac.uk m.hepple,i.roberts,saggion Department of Computer Science University of Sheffield, UK 1 Introduction The experiments detailed in this paper are a continuation of the experiments started as part of the work undertaken in preparation for participation in the TREC 2003 QA evaluations as documented in Gaizauskas et al. [2003]. Our main experiments for TREC 2004 were concerned with investigating: a) alternative approaches to information retrieval (IR) for question answering b) alternative approaches to answer extraction for list and factoid questions, and c) alternative approaches answering definitional or ‘other’ questions. In each of these three areas we have developed two principal alternatives, each of which has variants. Given the TREC limit of three test runs per site, we have not been able to evaluate properly all combinations of these approaches. Consequently, the systems we submitted only give a partial picture of work carried out, and further evaluation is underway. In the following we describe the major alternatives we have been exploring in these three areas and present the formal test results for the system combinations we submitted. 2 Information Retrieval for Question Anwering Like many others, we approach the QA task by first carrying out a retrieval step whose goal is to retrieve documents likely to contain answers to factoid, list or nugget-requesting questions. Analysis of previous year’s TREC results had convinced us that poor performance in this first step was having a serious negative impact on overall QA performance. Attracted by the relative transparency of boolean approaches to IR and