2_linear_classifiers.pptVIP

The Problem: Consider a two class task with ω1, ω2 LINEAR CLASSIFIERS Hence: The Perceptron Algorithm Assume linearly separable classes, i.e., The casefalls under the above formulation, since Our goal: Compute a solution, i.e., a hyperplane w,so that The steps Define a cost function to be minimized. Choose an algorithm to minimize the cost function. The minimum corresponds to a solution. The Cost Function Where Y is the subset of the vectors wrongly classified by w. When Y=O (empty set) a solution is achieved and J(w) is piecewise linear (WHY?) The Algorithm The philosophy of the gradient descent is adopted. Wherever valid  This is the celebrated Perceptron Algorithm. w An example: The perceptron algorithm converges in a finite number of iteration steps to a solution if A useful variant of the perceptron algorithm It is a reward and punishment type of algorithm. The perceptron It is a learning machine that learns from the training vectors via the perceptron algorithm. The network is called perceptron or neuron. Example: At some stage t the perceptron algorithm results in The corresponding hyperplane is ρ=0.7 Least Squares Methods If classes are linearly separable, the perceptron output results in If classes are NOT linearly separable, we shall compute the weights, , so that the difference between The actual output of the classifier, , and The desired outputs, e.g., to be SMALL. SMALL, in the mean square error sense, means to choose so that the cost function: Minimizing where Rx is the autocorrelation matrix and the crosscorrelation vector. Multi-class generalization The goal is to compute M linear discriminant functions: according to the MSE. Adopt as desired responses yi: Let And the matrix The goal is to compute : The above is equivalent to a number M of MSE minimization problems. That is: Design each so that its desired output is 1 for and 0 for any other class. Remark: