Field Theoretical Analysis of Online Learning of Probability Distributions.pdfVIP

Field Theoretical Analysis of On-line Learning of Probability Distributions∗ 9 9 9 Toshiaki Aida 1 Department of Physics, Tokyo Institute of Technology,Oh-okayama, Meguro-ku, Tokyo, Japan Tokyo Metropolitan College of Aeronautical Engineering, Minami-senjyu, Arakawa-ku, Tokyo, Japan v o (February 1, 2008) N 0 points x , x , . . . , x , which are drawn indepen- 1 2 N 3 On-line learning of probability distributions is an- dently from an unknown probability distribution ] alyzed from the field theoretical point of view. We Q∗ (x). How can we infer the distribution, which can obtain an optimal on-line learning algorithm, since n is specified by an infinite number of parameters in renormalization group enables us to control the num- n general? A finite number of sample points do not - ber of degrees of freedom of a system according to ∗ s the number of examples. We do not learn parameters enable us to determine Q (x) uniquely, but only i allows us to make a probabilistic description. We d of a model, but probability distributions themselves. t. Therefore, the algorithm requires no a priori knowl- start from the probability of a distribution Q(x), a edge of a model.