Dependency Parsing by Belief Propagation.pptVIP

Outline Why use joint models? Making big joint models tractable:Approximate inference and training by loopy belief propagation Open questions: Semi-supervised training of joint models The standard story Some running examples Semi-supervised learning Why would knowing p(x) help you learn p(y|x) ?? Semi-supervised learning Why would knowing p(x) help you learn p(y|x) ?? NLP + ML = ??? Each task in a vacuum? Solved tasks help later ones? (e.g, pipeline) Feedback? Feedback? A later step benefits from many earlier ones? A later step benefits from many earlier ones? We end up with a Markov Random Field (MRF) Variable-centric, not task-centric Task-centered view of the world Variable-centered view of the world Variable-centric, not task-centric Back to our (simpler!) running examples Parser projection Parser projection Parser projection Parser projection Parser projection Parses are not entirely isomorphic Dependency Relations Parser projection Parser projection Parser projection Parser projection Parser projection Parser projection Parser projection Parser projection Parser projection English does help us impute Chinese parse Which does help us train a monolingual Chinese parser (Could add a 3rd language …) (Could add world knowledge …) (Could add bilingual dictionary …) Dynamic Markov Random Field Back to our running examples Morphological paradigm Morphological paradigm Morphological paradigm as MRF # observations per form (fine-grained semisupervision) Preliminary results … Outline Why use joint models in NLP? Making big joint models tractable:Approximate inference and training by loopy belief propagation Open questions: Semisupervised training of joint models Key Idea! We’re using an MRF to coordinate the solutions to several NLP problems Each factor may be a whole NLP model over one or a few complex structured variables (strings, parses) Or equivalently, over many fine-grained variables (individual words, tags, links) Within a factor, use existing fas