Conditional Random Fields Probabilistic Models for Segmenting用于分割的条件随机场概率模型.pptVIP

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data John Lafferty Andrew McCallum Fernando Pereira Goal: Sequence segmentation and labeling Computational biology Computational linguistics Computer science Overview Generative models Conditional models Label bias problem Conditional random fields Experiments Generative Models HMMs and stochastic grammars Assign a joint probability to paired observation and label sequences Parameters are trained to maximize joint likelihood of training examples Generative Models Need to enumerate all possible observation sequences To ensure tractability of inference problem, must make strong independence assumptions (i.e., conditional independence given labels) Conditional models Specify probabilities of label sequences given an observation sequence Does not expend modeling effort on the observations which are fixed at test time Conditional probability can dependent on arbitrary, non-independent features of the observation sequence Example: MEMMs Maximum entropy Markov models Each source state has an exponential model that takes the observation feature as input and outputs a distribution over possible next states Weakness: Label bias problem Label Bias Problem Per-state normalization of transition scores implies “conservation of score mass” Bias towards states with fewer outgoing transitions State with single outgoing transition effectively ignores observation Solving Label Bias Collapse states, and delay branching until get a discriminating observation Not always possible or may lead to combinatorial explosion Solving Label Bias (cont’d) Start with fully-connected model and let training procedure figure out a good structure Precludes use of prior structure knowledge Overview Generative models Conditional models Label bias problem Conditional random fields Experiments Conditional Random Fields Undirected graph (random field) Construct conditional model p(Y|X) Does not explicitly model marginal p(X)