Conditional Random Fields条件随机场.pptVIP

Conditional Random Fields Rahul Gupta (KReSIT, IIT Bombay) Undirected models Graph representation of random variables Edges define conditional independence statements Graph separation criteria Markovian independence statements are enough P(S|V) = P(S|Nb(S)) Hammersley-Clifford Theorem Not all distributions satisfy Markovian properties The ones which do can be factorized as In practice, we assume Computing marginals We are interested in Marginals P(S) Conditionals P(S|S’) = P(S,S’)/P(S’) Same as computing marginals Labelings with highest joint probabilities Reuse the algorithm for computing marginals Only compute marginals for cliques Marginals in trees Sum-Product Algorithm Receive belief messages from children Pass consolidated belief to parent Node marginal prop. to Messages from children Node potential Marginals (contd.) Reuse messages for other marginals Running time dependent on tree diameter For best labeling, replace sum by max Store argmax along with max Junction-Tree algorithm Sum-Product fails to converge with cycles Make graph acyclic by merging cliques Run Sum-Product on transformed graph Local consistency Runtime exponential in max clique size Junction-Tree (contd.) Blindly arranging cliques is wrong Message Passing maintains only local consistency Need a ‘running intersection property’ Junction Tree (contd.) Junction-Tree cannot exist for un-triangulated graphs Triangulate the graph Max clique size may increase arbitrarily NP-hard to output the best triangulation ILP-based inferencing mc(yc) is 1 if clique c has value yc mc(yc)’s are mutually exclusive mc(yc) and mc’(yc’) are consistent for c’ ? c LP relaxation mc(yc) behave like marginal probabilities May admit invalid solutions if the graph is un-triangulated! Triangulation adds variables and constraints that keep the solution valid Approximate inferencing Sampling-based methods Variational methods Find upper and lower bounds to marginals Approximate algos when potentials a