Searching CiteSeer Metadata Using Nutch.pdf

Searching CiteSeer Metadata Using Nutch Larry Reeve for Dr. Lin INFO624, Winter 2005 Term Project Table of Contents Background………………………………………………………………………………………………………………………………………………… 2 Project Design……………………………………………………………………………………………………………………………………… 3 Evaluation………………………………………………………………………………………………………………………………………………… 10 Future Work……………………………………………………………………………………………………………………………………………… 15 References………………………………………………………………………………………………………………………………………………… 17 Appendix A - CiteSeer Metadata Record Example ………………………………………………… 18 Appendix B – Converted HTML Example …………………………………………………………………………… 19 Appendix C - Shell Script to Generate HTML Files from XML Metadata Files ………………………………………………………………………………………………………… 20 Appendix D - Parse XML Metadata and Generate HTML………………………………………… 21 Appendix E - Shell Script to Crawl the Generated HTML Files …………… 24 Appendix F - Nutch Parse Filter for CiteSeer Metadata …………………………… 25 Appendix G - Simple Parser to Split Author Name into Two Parts …… 28 Appendix H - Simple Parser to Split Date Field into Three Parts … 29 Appendix I - Nutch Indexing Filter for CiteSeer Metadata …………………… 30 Appendix J - Nutch Query Filter for CiteSeer Metadata……………………………… 32 Appendix K - Plugin Build File for Apache Ant…………………………………………………… 33 Appendix L - Nutch Plugin Directives for CiteSeer Plugin……………………… 34 Appendix M - JSP Source for Search Application………………………………………………… 35 Larry Reeve INFO624 - Winter 2005 Page 1 Background CiteSeer (/) is a scientific literature digital library that allows searching for scholarly publications by document or by citations. The search engine used by CiteSeer is based on keyword search. A problem with the keyword approach is that searche