牛津大学机器学习课件10.pdfVIP

Max-margin learning Nando de Freitas Outline of the lecture Max margin learning is an extremely powerful idea for learning features with auxiliary tasks, and then use these features to solve tasks with few data. The goal of this lecture is for you to learn Transfer, multi-task, and multi-instance learning Harnessing auxiliary tasks to learn features Matchings Preferences Corruption Formulations of multi-task learning Applications: Cross-lingual embeddings Relation learning Question answering Memory networks Embedding discrete objects in metric spaces Code Words Formulae Logical expressions Symbols DNA sequences Idea: learn embeddings (features) in one task and transfer these to solve new tasks [Kotzias, Denil NdF, 2014] Deep Multi-Instance Learning Deep Multi-Instance Learning Auxiliary tasks to learn features that can be transferred to learn tasks with few labels 1. Matchings [From machine learning to machine reasoning, Leon Bottou] Auxiliary tasks to learn features that can be transferred to learn tasks with few labels 2. Corruption [NLP almost from scratch, Ronan Collobert and colleagues] Auxiliary tasks to learn features that can be transferred to learn tasks with few labels 1. Preferences Max-margin formulations Max-margin formulations [A tutorial on energy based learning, Yann LeCun et al] Max-margin formulations Hinge Loss unconstrained formulation: Max-margin formulations Hinge loss layer Example: Bi-lingual word embeddings [Karl Hermann and Phil Blunsom, 2014] Siamese networks (Yann LeCun) Semi-supervised deep learning (Jason Weston et al) Applicati