Belief Propagation - Machine Learning Reading Group Brown 信念传播-机器学习阅读组棕色.pptVIP

3/1/2008 MLRG Belief Propagation on Markov Random Fields Aggeliki Tsoli Outline Graphical Models Markov Random Fields (MRFs) Belief Propagation Graphical Models Diagrams Nodes: random variables Edges: statistical dependencies among random variables Advantages: Better visualization conditional independence properties new models design Factorization Graphical Models types Directed causal relationships e.g. Bayesian networks Undirected no constraints imposed on causality of events (“weak dependencies”) Markov Random Fields (MRFs) Example MRF Application: Image Denoising Question: How can we retrieve the original image given the noisy one? MRF formulation Nodes For each pixel i, xi : latent variable (value in original image) yi : observed variable (value in noisy image) xi, yi ? {0,1} MRF formulation Edges xi,yi of each pixel i correlated local evidence function ?(xi,yi) E.g. ?(xi,yi) = 0.9 (if xi = yi) and ?(xi,yi) = 0.1 otherwise (10% noise) Neighboring pixels, similar value compatibility function ?(xi, xj) MRF formulation Question: What are the marginal distributions for xi, i = 1, …,n? Belief Propagation Goal: compute marginals of the latent nodes of underlying graphical model Attributes: iterative algorithm message passing between neighboring latent variables nodes Question: Can it also be applied to directed graphs? Answer: Yes, but here we will apply it to MRFs Belief Propagation Algorithm Select random neighboring latent nodes xi, xj Send message mi?j from xi to xj Update belief about marginal distribution at node xj Go to step 1, until convergence How is convergence defined? Step 2: Message Passing Message mi?j from xi to xj : what node xi thinks about the marginal distribution of xj Step 3: Belief Update Belief Propagation Algorithm Select random neighboring latent nodes xi, xj Send message mi?j from xi to xj Update belief about marginal distribution at node xj Go to step 1, until convergence Example - Compute belief at