Gaussian_Processes_in_Machine_Learning.ppt

Gaussian Processes in Machine Learning Gerhard Neumann, Seminar F, WS 05/06 Outline of the talk Gaussian Processes (GP) [ma05, rs03] Bayesian Inference GP for regression Optimizing the hyperparameters Applications GP Latent Variable Models [la04] GP Dynamical Models [wa05] GP: Introduction Gaussian Processes: Definition: A GP is a collection of random variables, any finite number of which have joint Gaussian Distribution Distribution over functions: Gaussian Distribution: over vectors Nonlinear Regression: XN … Data Points tN … Target Vector Infer Nonlinear parameterized function, y(x;w), predict values tN+1 for new data points xN+1 E.g. Fixed Basis Functions Bayesian Inference of the parameters Posterior propability of the parameters: Probability that the observed data points have been generated by y(x;w) Often separable Gaussian distribution is used Each data point ti differing from y(xi;w) by additive noise priors on the weights Prediction is made by marginalizing over the parameters Integral is hard to calculate Sample parameters w from the distribution with Markov chain Monte Carlo techniques Or Approximate with a Gaussian Distribution Bayesian Inference: Simple Example GP: is a Gaussian distribution Example: H Fixed Basis functions, N input points Prior on w: Calculate prior for y(x) : Prior for the target values generated from y(x;w) + noise: Covariance Matrix: Covariance Function Predicting Data Infer tN+1 given tN : Simple, because conditional distribution is also a Gaussian Use incremental form of Predicting Data We can rewrite this equation Use partitioned inverse equations to get from Predictive mean: Usually used for the interpolation Uncertainty in the result : Bayesian Inference: Simple Example How does the covariance matrix look like? Usually N H: Q has not full rank, but C has (due to the addition of I) Simple Example: 10 RBF functions, uniformly distributed o