Learning to Match and Cluster Large High-Dimensional Data Sets.pdfVIP

Learning to Match and Cluster Large High-Dimensional Data Sets For Data Integration William W. Cohen Jacob Richman WhizBang Labs WhizBang Labs 4616 Henry St. 4616 Henry St. Pittsburgh, PA 15213 Pittsburgh, PA 15213 william@ jsr@ ABSTRACT global object identifiers are rare. Part of the process of data integration is determining which Previous publications in using textual similarity for data sets of identifiers refer to the same real-world entities. In integration have considered a number of related tasks. Al- integrating databases found on the Web or obtained by us- though the terminology is not completely standardized, in ing information extraction methods, it is often possible to this paper we define entity-name matching as the task of solve this problem by exploiting similarities in the textual taking two lists of entity names from two different sources names used for objects in different databases. In this paper and determining which pairs of names are co-referent (i.e., we describe techniques for clustering and matching identifier refer to the same real-world entity). We define entity-name names that are both scalable and adaptive, in the sense that clustering as the task of taking a single list of entity names they can be trained to obtain better performance in a par- and assigning entity names to cl