集成学习.ppt

Ensemble Learning（集成学习）;Outline;What’s is Machine Learning;美国航空航天局JPL实验室的科学家在《Science》（2001年9月）上撰文指出：机器学习对科学研究的整个过程正起到越来越大的支持作用，……，该领域在今后的若干年内将取得稳定而快速的发展;No Free Lunch Theorem;No Free Lunch Theorem;It is the assumptions about the learning algorithm that are important Even popular algorithms will perform poorly on some problems, where the learning algorithm and data distribution do not match well In practice, experience with a broad range of techniques is the best insurance for solving arbitrary new classification problems;;;;;;;;;;;;;;;;;;;;Bias and Variance;Bias and Variance;Motivation;*;*;*;集成学习(Ensemble Learning)是一种机器学习范式，它使用多个学习器来解决同一个问题;Example: Weather Forecast;Majority vote Suppose we have 5 completely independent classifiers… If accuracy is 70% for each 10 (.7^3)(.3^2)+5(.7^4)(.3)+(.7^5) 83.7% majority vote accuracy 101 such classifiers 99.9% majority vote accuracy ;*;Ensemble learning;Define the misfit of function as The mean square error;The average mean square error Ensemble regression function ;Ensemble learning;Ensemble learning;Ensemble learning;既然多个学习器的集成比单个学习器更好，那么是不是学习器越多越好?;选择性集成;选择性集成;选择性集成; 提出了选择性集成(Selective Ensemble) 证明了 “Many Could be Better Than All” Theorem 在有一组个体学习器可用时，从中选择一部分进行集成，可能比用所有个体学习器进行集成更好;选择性集成思想的一般性：利用多个个体，并对个体进行选择，可以获得更好的结果;*;*;*;*;*;*;*;*;*;AdaBoost Its Applications;Introduction;AdaBoost Concept;Weaker Classifiers;The Strong Classifiers;AdaBoost Its Applications;The AdaBoost Algorithm;The AdaBoost Algorithm;Boosting illustration;Boosting illustration;Boosting illustration;Boosting illustration;Boosting illustration;Boosting illustration;AdaBoost Its Applications;The AdaBoost Algorithm;The AdaBoost Algorithm;Goal;Goal;;;Final classifier:;;with;;with;;;with;Netflix;Since October 2006;Supervised learning task Training data is a set of users and ratings (1,2,3,4,5 stars) those users have given to movies. Construct a classifier that given a user and an unrated movie, correctly classifies that movie as either 1, 2,