LNAI 4212 - Transductive Gaussian Process Regression with.pdfVIP

Transductive Gaussian Process Regression with Automatic Model Selection 1 1 2 3 Quoc V. Le , Alex J. Smola , Thomas G¨artner , and Yasemin Altun 1 RSISE, Australian National University, 0200 ACT, Australia Statistical Machine Learning Program, National ICT Australia, 0200 ACT, Australia 2 Fraunhofer IAIS, Schloß Birlinghoven, 53754 Sankt Augustin, Germany 3 Toyota Technological Institute at Chicago, Chicago IL 60632, USA Abstract. In contrast to the standard inductive inference setting of predictive ma- chine learning, in real world learning problems often the test instances are already available at training time. Transductive inference tries to improve the predictive accuracy of learning algorithms by making use of the information contained in these test instances. Although this description of transductive inference applies to predictive learning problems in general, most transductive approaches consider the case of classification only. In this paper we introduce a transductive variant of Gaussian process regression with automatic model selection, based on approxi- mate moment matching between training and test data. Empirical results show the feasibility and competitiveness of this approach. 1 Introduction Machine learning research mostly concentrates on estimating an underlying unknown conditional or functional dependence of a target property on some other variables. This estimate is based on a set of training instances, respecting this dependency. It is then usually applied to test instances for which the target property has not been observed. This setting is known as supervised learning or inductive inference. The downside of such algorithms is that they igno