General Loss Bounds for Universal Sequence Prediction.pdf

a r X i v : c s / 0 1 0 1 0 1 9 v 2 [ c s .A I ] 1 9 S e p 2 0 0 1 General Loss Bounds for Universal Sequence Prediction Marcus Hutter IDSIA, Galleria 2, CH-6928 Manno-Lugano, Switzerland marcus@idsia.ch http://www.idsia.ch/～marcus Technical Report IDSIA-03-01, 10 April 2001 Keywords Bayesian and deterministic prediction; general loss function; Solomonoff induction; Kolmogorov com- plexity; leaning; universal probability; loss bounds; games of chance; partial and delayed prediction; classification. Abstract The Bayesian framework is ideally suited for induc- tion problems. The probability of observing xt at time t, given past observations x1...xt?1 can be com- puted with Bayes’ rule if the true distribution μ of the sequences x1x2x3... is known. The problem, however, is that in many cases one does not even have a reasonable estimate of the true distribution. In order to overcome this problem a universal dis- tribution ξ is defined as a weighted sum of distri- butions μi ∈ M , where M is any countable set of distributions including μ. This is a generalization of Solomonoff induction, in which M is the set of all enumerable semi-measures. Systems which predict yt, given x1...xt?1 and which receive loss lxtyt if xt is the true next symbol of the sequence are considered. It is proven that using the universal ξ as a prior is nearly as good as using the unknown true distribu- tion μ. Furthermore, games of chance, defined as a sequence of bets, observations, and rewards are studied. The time needed to reach the winning zone is bounded in terms of the relative entropy of μ and ξ. Extensions to arbitrary alphabets, partial and delayed prediction, and more active systems are dis- cussed. 1 Introduction 1.1 Induction Many problems are of induction type, in which statements about the future have to be made, based on past obser- vations. What is the probability of rain tomorrow, given the weather observations of the last few days? Is the Dow Jones likely to rise tomorrow