Supervised+Learning+Methods.pptxVIP

SupervisedLearningMethodslanyanyan@WhatisSupervisedLearning?Supervisedlearning isthemachinelearningtaskofinferringafunctionfromlabeledtrainingdata.Thetrainingdata consistofasetof trainingexamples.Insupervisedlearning,eachexampleisa pairconsistingofaninputobject(typicallyavector)andadesiredoutputvalue(alsocalledthe supervisorysignal).Asupervisedlearningalgorithmanalyzesthetrainingdataandproducesaninferredfunction,whichcanbeusedformappingnewexamples.Anoptimalscenariowillallowforthealgorithmtocorrectlydeterminetheclasslabelsforunseeninstances.Thisrequiresthelearningalgorithmtogeneralizefromthetrainingdatatounseensituationsinareasonableway.RegressionMethodsRegressionLinearRegressionwithoneVariableExample:Supposewehaveadatasetgivingthelivingareaandpricesof47housesfromPortland,Oregon:MutivariateLinearRegressionClosedFormofSolutionLeastMeanSquaresAlgorithmOptimizationProblemGradientDecent:Updaterule:BatchGradientDescentStochasticGradientDescentSGDvs.BGDLinearRegressionwithNonlinearBasisClosedFormofSolutionforNonlinearBasisProbabilisticExplanation:MLELog-LikelihoodFunctionsumofsquareerrorRegularizedLMSasMAPPosteriorDistributionofParametersExample:StraightLineFitDataGenerationMLEvs.MAPGeometryofLMSLocallyWeightedLinearRegressionThedatadoesnotreallylieonstraightline.Assumingthereissufficienttrainingdata,makesthechoiceoffeatureslesscritical5minBreakClassificationMethodsClassificationProblemDiscriminantFunction:PerceptronF.RosenblattPerceptronCriterionPerceptronAlgorithmPerceptronLearningIllustrationDisadvantagesofPerceptronDoesnotconvergeifclassesarenotlinearlyseparableDoesnotprovideprobabilisticoutputDiscriminativeModel:LogisticRegressionLogisticfunctionSigmoidfunctionMaximumLikelihoodEstimatorProbabilityDistributionLikelihood:Maximizetheloglikelihood:Gradient:SGD:BernoulliMulti-ClassLogisticRegressionGeneralizedBernoulliMulti-ClassLikelihoodFunctionOptimizationforMulti-ClassLRCrossEntropyLossFunctionApplicationofLogisticRegression核心Generative:GaussianDiscriminativeAnalysis