Machine learning for targeted display advertising- Transfer learning in action.pdf

Mach Learn (2014) 95:103–127 DOI 10.1007/s10994-013-5375-2 Machine learning for targeted display advertising: transfer learning in action C. Perlich · B. Dalessandro · T. Raeder · O. Stitelman · F. Provost Received: 19 November 2012 / Accepted: 29 April 2013 / Published online: 30 May 2013 ? The Author(s) 2013 Abstract This paper presents the design of a fully deployed multistage transfer learning system for targeted display advertising, highlighting the important role of problem formula- tion and the sampling of data from distributions different from that of the target environment. Notably, the machine learning system itself is deployed and has been in continual use for years for thousands of advertising campaigns—in contrast to the more common case where predictive models are built outside the system, curated, and then deployed. In this domain, acquiring sufficient data for training from the ideal sampling distribution is prohibitively expensive. Instead, data are drawn from surrogate distributions and learning tasks, and then transferred to the target task. We present the design of the transfer learning system We then present a detailed experimental evaluation, showing that the different transfer stages indeed each add value. We also present production results across a variety of advertising clients from a variety of industries, illustrating the performance of the system in use. We close the paper with a collection of lessons learned from over half a decade of research and develop- ment on this complex, deployed, and intensely used machine learning system. Keywords Transfer learning · Display advertising · Predictive modeling Editors: Kiri Wagstaff and Cynthia Rudin. C. Perlich () · B. Dalessandro · T. Raeder · O. Stitelman · F. Provost M6D Research, 37 E. 18th St., New York, NY, USA e-mail: claudia@ B. Dalessandro e-mail: briand@ T. Raeder e-mail: troy@ O. Stitelman e-mail: ori@ F. Provost Leonard N. Stern School of Business, New York University, 44 W. 4th St.,