TheClassicTF-IDFVectorSpaceModel-TheClassicTF-IDF.PDFVIP

The Classic TF-IDF Vector Space Model Abstract – This is Part 3 of an introductory tutorial series on Term Vector Theory. The classic term frequency-inverse document frequency model or TF-IDF, is discussed. Keywords: vectors, term vector theory, term frequency, inverse document frequency, tf-idf, cosine similarity Published: 10-27-2006; Updated: 03-19-2016 ? E. Garcia, PhD; admin@ Note: This article is part of a legacy series that the author published circa 2006 at , now a search engine site. It is now republished in pdf format here at , with its content edited and updated. The original articles can be found referenced in online research publications on IR and elsewhere. Introduction In Parts 1 and 2 of this series on vector space models for Information Retrieval (IR) we mentioned that an index of terms is constructed by extracting n number of unique terms from a collection of D number of documents. Local, Li,j, and global, Gi, weights can then be assigned to index terms appearing in documents and queries (Garcia, 2016a; 2016b). For instance, the weight of an index term i present in a document j can be computed as (1) A normalization weight, Nj, is some times included. Its meaning is discussed later in this tutorial. One can find in the IR literature (Chisholm Kolda, 1999; Lee, Chuang, Seamons, 1997; Baeza-Yates Ribeiro-Neto, 1999; Grossman Frieder, 2004; Rijsbergen, 2004) different weighting schemes based on (1). For instance, in the Binary Model (BNRY), Li,j is defined based on the presence or absence of an index term i in a document j, regardless of its frequency, fi,j, and global weight. (2)