Quantitative evaluation of passage retrieval algorithms for question answering：（通过检索算法的定量评价问题回答）.pdfVIP

Quantitative Evaluation of Passage Retrieval Algorithms for Question Answering Stefanie Tellex, Boris Katz, Jimmy Lin, Aaron Fernandes, and Gregory Marton MIT Artificial Intelligence Laboratory Cambridge, Massachusetts, USA {stefie10,boris,jimmylin,adfernan,gremio}@ ABSTRACT Over the past few years, the question answering tracks at Passage retrieval is an important component common to the Text Retrieval Conferences (TREC) [25, 23, 24] have many question answering systems. Because most evalu- brought formal and rigorous evaluation methodologies to ations of question answering systems focus on end-to-end bear on the question answering task. Although the impor- performance, comparison of common components becomes tance of these evaluations cannot be denied, they measure only the end-to-end performance of complex systems that difficult. To address this shortcoming, we present a quan- titative evaluation of various passage retrieval algorithms typically involve a combination of information retrieval, in- for question answering, implemented in a framework called formation extraction, and natural language processing tech- Pauchok. We present three important findings: Boolean nologies. Without further ablation studies (e.g., [16, 13]), it querying schemes perform well in the question answering is difficult to untangle the performance contributions of the task. The performance differen