机器学习与概率图模型中科院自动化所系列报告(王立威).pptVIP

Thm.4: Exact and approximate MAP inference are all hard. Conclusion: The worst-case complexity of the inferences, both exact and approximate are NP-hard. * 第六十二页，共一百二十三页。 Exact Inference Algorithms * 第六十三页，共一百二十三页。 The relation between BN and MRF From BN to MRF (factors) * BN: MRF: BN -- MRF: 第六十四页，共一百二十三页。 From BN to MRF (graphs) Moral graph （联姻图） Delete the directions for edges; Connecting the parents. * 第六十五页，共一百二十三页。 Exact inference algorithms: The variable elimination algorithm. Belief propagation: the sum-product algorithm. Belief update. Why study exact inference algorithms? Gain intuition and insight for developing useful approximate algorithms. * 第六十六页，共一百二十三页。 The variable elimination algorithm query: distribution: solve: * 第六十七页，共一百二十三页。 Variable elimination---dynamic programming * 第六十八页，共一百二十三页。 * Step 1: Step 2: (g,e) (a,e,f) 第六十九页，共一百二十三页。 * Step 3: Step 4: (a,c,d,e) (a,c,d) 第七十页，共一百二十三页。 * Step 5: Step 6: (a,b,c) (a,b) 第七十一页，共一百二十三页。 Message Passing: the SP algorithm Variable elimination induced clique tree * Running intersection property! Family preserving property: factors scope 第七十二页，共一百二十三页。 General message passing from clique * : original terms (potential) of Variable elimination as message passing on clique tree: 第七十三页，共一百二十三页。 The Sum-Product belief propagation algorithm: Construct a clique tree that satisfies the family preserving and running intersection properties. Choose clique where the query variable lies in as the root. Message passing from the leaves. After all messages arrive at the root, sum over all other variables in the root clique other than the query variable. * Thm: The Sum-Product algorithm always gives the correct answer for any clique tree! 第七十四页，共一百二十三页。 What if we want to compute k queries? Message passing k times? No! Message can be reused. For each edge in the clique tree, twice is enough, one for each direction (up-down) * 第七十五页，共一百二十三页。 After message passing on all edges in two directions,