苹果AI负责人演讲：深度生成模型定量评估.pdf

On the Quantave Evaluaon of Deep Generave Models Russ Salakhutdinov Machine Learning Department Carnegie Mellon University Joint work with Yuhuai Wu, Yuri Burda, Roger Grosse Talk Roadmap •  Introducon •  Annealed Importance Sampling (AIS) •  Reverse AIS Esmator for Undirected Graphical Models •  Evaluang Directed, Decoder-Based Generave Models References •  Accurate and Conservave Esmates of MRF Log-likelihood using Reverse Annealing Yuri Burda, Roger Grosse, Ruslan Salakhutdinov, AI and Stascs, 2015 •  On the Quantave Analysis of Decoder-Based Generave Models Yuhuai Wu, Yuri Burda, Ruslan Salakhutdinov, Roger Grosse, ICLR 2017 Unsupervised Learning Non-probabilisc Models Probabilisc (Generave) Ø  Sparse Coding Models Ø  Autoencoders Ø  Others (e.g. k-means) Non-Tractable Models Tractable Models Ø  Generave Adversarial Ø  Fully observed Ø  Boltzmann Machines Networks Belief Nets Ø  Variaonal Ø  Moment Matching Ø  NADE Autoencoders Networks Ø  PixelRNN Ø  Helmholtz Machines Ø  Many others… Explicit Density p(x) Implicit Density Directed vs. Undirected Models Helmholtz Machine / Variational Autoencoder Deep Boltzmann Machine Generave Approximate Process Inference h3 h3