Multivariate Information Bottleneck HUJI多元信息瓶颈胡.pptVIP

Multivariate Information Bottleneck Multivariate Information Bottleneck - Preview Multivariate IB – introduction Second half starts here… Maybe a temporary summary – a well defined method (formulated as a variational principle)… 3 different algorithmic approaches… however – it was limited for a specific optimization problem… but we could think of other problems (e.g. symmetric)… and in the following we will describe a lift-up of the first half for dealing with a much more rich family of problems… still work in progress… Multivariate IB – introduction (cont.) (a few words about …) Bayesian Networks Multi-information and Bayes nets Original IB through Bayes net formulation Alternative formulation: preliminaries Alternative formulation for original IB Comparing the two principles Comparing the two principles (cont.) Beyond the original IB[Slonim, Friedman, Tishby] A simple example: Symmetric IB A multivariate formal optimal solution Multivariate IB algorithms – example for aIB[Slonim, Friedman, Tishby, 2002] Symmetric aIB compression: documents, words Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric IB through Deterministic Annealing Symmetric aIB compression: genes, samples Symmetric aIB compression: genes, samples Another example: parallel IB Another example: parallel IB (cont.) Another example: parallel IB (cont.) Parallel IB: solution Parallel sIB: Text analysis results Parallel sIB :Gene Expression data results Another Example: Triplet IB Triplet IB Data Summary Minimize dependencies Maximize dependencies Effective distortion: Data: ~1,500 “documents” taken from E. R. Burroughs: The Beasts of Tarzan The Gods of Mars R. Kipling: