Introduction to Markov Chain Monte Carlo techniques.ppt

Introduction to Sampling based inference and MCMC Ata Kaban School of Computer Science The University of Birmingham The problem Up till now we were trying to solve search problems (search for optima of functions, search for NN structures, search for solution to various problems) Today we try to:- Compute volumes Averages, expectations, integrals Simulate a sample from a distribution of given shape Some analogies with EA in that we work with ‘samples’ or ‘populations’ The Monte Carlo principle p(x): a target density defined over a high-dimensional space (e.g. the space of all possible configurations of a system under study) The idea of Monte Carlo techniques is to draw a set of (iid) samples {x1,…,xN} from p in order to approximate p with the empirical distribution Using these samples we can approximate integrals I(f) (or v large sums) with tractable sums that converge (as the number of samples grows) to I(f) Importance sampling Target density p(x) known up to a constant Task: compute Idea: Introduce an arbitrary proposal density that includes the support of p. Then: Sample from q instead of p Weight the samples according to their ‘importance’ It also implies that p(x) is approximated by Efficiency depends on a ‘good’ choice of q. Sequential Monte Carlo Sequential: Real time processing Dealing with non-stationarity Not having to store the data Goal: estimate the distrib of ‘hidden’ trajectories We observe yt at each time t We have a model: Initial distribution: Dynamic model: Measurement model: Can define a proposal distribution: Then the importance weights are: Obs. Simplifying choice for proposal distribution:Then: Applications Computer vision Object tracking demo [BlakeIsard] Speech audio enhancement Web statistics estimation Regression classification Global maximization of MLPs [Freitas et al] Bayesian networks Details in Gilks et al book (in the School library) Genetics molecular biology Robotics, etc. M Isard A Blake: CONDENSATION – conditio