InformationExtractiononRealEstateRentalClassifieds.pptVIP

Information Extraction on Real Estate Rental Classifieds Eddy Hartanto Ryohei Takahashi Overview We want to extract 10 fields: Approach Hand labeled test set as precision and recall computation base Pattern matching approach with Rapier Statistical approach using HMM with different structures Demo … Hidden Markov Models We consider three different HMM structures We train one HMM per field Words in postings are output symbols of HMM Hexagons represent target states, which emit the relevant words for that field Training Data We use a randomly-selected set of 110 postings to use as the training data We manually label which words in each posting are relevant to each of the 10 fields HMM Structure #1 A single prefix state and single suffix state Prefixes and suffixes can be of arbitrary length HMM Structure #2 Varying numbers of prefix, suffix, and target states HMM Structure #3 Varying numbers of prefix, suffix, and target states Prefixes and suffixes are fixed in length Cross-Validation We use cross-validation to find the optimal number of prefix, suffix, and target states Preventing Underflow Postings are hundreds of words long Forward and backward probabilities become incredibly small = underflow To avoid underflow, we normalize the forward probabilities: instead of Smoothing We perform add-one smoothing for the emission probabilities: Rapier Rapier automatically learns rules to extract fields from training examples We use the same 110 training postings as for the HMMs Data Preparation Sentence Splitter (Cognitive Computation Group at UIUC, /~cogcomp/tools.php): puts one sentence on each line Stanford Tagger (Stanford NLP Group, /software/tagger.shtml): tags each word with part of speech We then manually create a template file for each of the files, with the information for the 10 fields filled in Test Data We use a randomly-selected set of 100 postings to use as the test data We manually label these 100 postings with the fields Rapier Results We use Rapier’s “test2