TextDatabase.pptVIP

Text Database Text Fuzzy Match Keyword/Key Phrase Extraction Fuzzy Match Applications: Music Retrieval Term Association Analysis Text Categorization Text Summarization, Clustering, and Its Applications Values of This Course Information access service is highly profitable Text database techniques are its core technologies Text Fuzzy Match Applications Text search, text highlight duplicate detection, record linkage Levels of Text Fuzzy Match Phonic: homonyms, transliteration match Symbolic: most common type of text match Semantic: knowledge bases are often needed Text Fuzzy Match – Phonic (En) For English, use Soundex code a hashing system for English words used by the United States Census Bureau by Robert C. Russell (U.S. patent 1,261,167 on April 2, 1918) Soundex Algorithm begin with the first letter of a word followed by a three-digit (1-6) code that represents the (first three) remaining consonants If the word is too short to generate 3 numbers, 0 is added as needed. If the generated code is longer than 3 numbers, the extra are thrown away. Text Fuzzy Match – Phonic (En) Examples: perl -e use Text::Soundex; print soundex(anything) Ans: A535 References: /alan/numbers/soundex.html /cgi-bin/Genea/soundex.sh Applications: Yuen-Hsien Tseng, Automatic Cataloguing and Searching for Retrospective Data by Use of OCR Text, Journal of the American Society for Information Science and Technology, Vol. 52, No. 5, April 2001, pp. 378-390. (SSCI, SCI, EI) Text Fuzzy Match – Phonic (Ch) For Chinese, convert each character into its phonograms or pinyins match their phonograms or pinyins instead of their characters Applications Transliteration match:克麗斯汀 vs 克莉思汀 Traditional/simplified character match Example: D:\demo\ir\sam.libperl -s Fuzzyhighlight.pm 國內大型電腦集團赴大陸投資升溫 國內電腦廠商赴大陸頭姿 1 Ans: font color=red國內大型電腦集團赴大陸投資/font升溫 D:\demo\ir\sam.libperl -s Fuzzyhighlight.pm 國內大型電腦集團赴大陸投資升溫 國內電腦廠商,赴大陸投資,頭姿 1 Ans: 國內大型電腦集團font color=red赴大陸font color=red投資/font/font升溫 Text Match (Na?ve) Simpl