PAC-Bayes analysis of multi-view learning.pdfVIP

Information Fusion 35 (2017) 117–131 Contents lists available at ScienceDirect Information Fusion journal homepage: /locate/inffus PAC-Bayes analysis of multi-view learning Shiliang Sun a,?, John Shawe-Taylor b, Liang Mao a a Department of Computer Science and Technology, East China Normal University, 500 Dongchuan Road, Shanghai 200241, P. R. China b Department of Computer Science, University College London, Gower Street, London WC1E 6BT, United Kingdom article info Article history: Received 3 August 2016 Revised 22 September 2016 Accepted 27 September 2016 Available online 28 September 2016 Keywords: PAC-Bayes bound Statistical learning theory Support vector machine Multi-view learning abstract Multi-view learning is a widely applicable research direction. This paper presents eight PAC-Bayes bounds to analyze the generalization performance of multi-view classi?ers. These bounds adopt data dependent Gaussian priors which emphasize classi?ers with high view agreements. The center of the prior for the ?rst two bounds is the origin, while the center of the prior for the third and fourth bounds is given by a data dependent vector. An important technique to obtain these bounds is two derived logarithmic determinant inequalities whose difference lies in whether the dimensionality of data is involved. The centers of the ?fth and sixth bounds are calculated on a separate subset of the training set. The last two bounds use unlabeled data to represent view agreements and are thus applicable to semi-supervised multi-view learning. We evaluate all the presented multi-view PAC-Bayes bounds on benchmark data and compare them with previous single-view PAC-Bayes bounds. The usefulness and performance of the multi-view bounds are discussed. ? 2016 Elsevier B.V. All rights reserved. 1. Introduction Multi-view learning is a promising research direction with prevalent applicability [1]. For instance, in multimedia content understanding, multimedia segments can be described by both the