patwardhan.pptVIP

Incorporating Dictionary and Corpus Information into a Context Vector Measure of Semantic Relatedness Siddharth Patwardhan Advisor: Ted Pedersen 07/18/2003 Semantic Relatedness Is needle more related to thread than it is to pie? Humans agree on the relatedness of most word pairs. Miller and Charles (1991) suggest that relatedness is based on the overlap of contextual representation of words. Measuring Relatedness Automatic measures that attempt to imitate the human perception of relatedness of words and concepts. Number of measures of relatedness, based on WordNet and corpus data have been proposed. In this thesis, we compare various of measures of semantic relatedness. WordNet Semantic network. Nodes represents real world concepts. Rich network of relationships between these concepts. Relationships such as “car is a kind of vehicle”, “high opposite of low”, etc exist. Node = Synonyms + Definition (gloss) WordNet – Schematic WordNet – Is a Hierarchy WordNet – Overview Four parts of speech – nouns, verbs, adjectives and adverbs. ~ 111,400 concepts. ~ 13 types of relationships. 9 noun is a hierarchies with an average depth of 13. 628 verb is a hierarchies with an average depth of 2. The Adapted Lesk Algorithm Performs Word Sense Disambiguation (Banerjee and Pedersen 2002). Uses the overlaps of dictionary definitions of word senses to determine sense of the target. Basic hypothesis – correct sense of the target word is most related to the senses of the words in its context. Overlaps measure the relatedness. Our Extension of Adapted Lesk Use any measure in place of gloss overlaps and perform WSD. Extended Gloss Overlaps is a measure of semantic relatedness. Extended Gloss Overlaps Measure Gloss Overlaps – Scoring Leacock-Chodorow Measure (1998) Based on simple edge counts in the is a hierarchy of WordNet. Deals with nouns only. The path length is scaled by the depth of the taxonomy. The Resnik Measure (1995) Deals with nouns only and is based on the is a hierarchy of