Lecture2Modeling-AlexanderGelbukh.pptVIP

Lecture 1 Special Topics in Computer Science Advanced Topics in Information Retrieval Chapter 2: Modeling Alexander Gelbukh www.G Previous chapter User Information Need Vague Semantic, not formal Document Relevance Order, not retrieve Huge amount of information Efficiency concerns Tradeoffs Art more than science Modeling Still science: computation is formal No good methods to work with (vague) semantics Thus, simplify to get a (formal) model Develop (precise) math over this (simple) model Why math if the model is not precise (simplified)? phenomenon ? model = step 1 = step 2 = ... = result math phenomenon ? model ? step 1 ? step 2 ? ... ? ?! Modeling Substitute a complex real phenomenon with a simple model, which you can measure and manipulate formally Keep only important properties (for this application) Do this with text: ? Modeling in IR: idea Tag documents with fields As in a (relational) DB: customer = {name, age, address} Unlike DB, very many fields: individual words! E.g., bag of words: {word1, word2, ...}: {3, 5, 0, 0, 2, ...} Define a similarity measure between query and such a record (Unlike DB) Rank (order), not retrieve (yes/no) Justify your model (optional, but nice) Develop math and algorithms for fast access as relational algebra in DB Taxonomy of IR systems Aspects of an IR system IR model Boolean, Vector, Probabilistic Logical view of documents Full text, bag of words, ... User task retrieval, browsing Independent, though some are more compatible Appropriate models Characterization of an IR model D = {dj}, collection of formal representations of docs e.g., keyword vectors Q = {qi}, possible formal representations of user information need (queries) F, framework for modeling these two: reason for the next R(qi,dj): Q ? D ? R, ranking function defines ordering Specific IR models IR models Classical Boolean Vector Probabilistic (clear ideas, but some disadvantages) Refined Each one with