IndexingandRepresentationTheVectorSpaceModel.pptVIP

Slide courtesy Ray Larson Indexing and Representation:The Vector Space Model Document represented by a vector of terms Words (or word stems) Phrases (e.g. computer science) Removes words on “stop list” Documents aren’t about “the” Often assumed that terms are uncorrelated. Correlations between term vectors implies a similarity between documents. For efficiency, an inverted index of terms is often stored. Document RepresentationWhat values to use for terms Boolean (term present /absent) tf (term frequency) - Count of times term occurs in document. The more times a term t occurs in document