sma5505projectfinalreport.docVIP

SMA 5505 Project Final Report Parallel SMO for Training Support Vector Machines May 2003 Deng Kun Lee Yih Asankha Perera Table of Contents 1 Introduction 3 2 Support Vector Machines (SVM) 4 2.1 Introduction to SVM 4 2.2 General Problem Formulation 5 2.3 The Karush-Kahn-Tucker (KKT) Condition 6 2.4 A Brief Survey of SVM Algorithms 7 3 Sequential Minimal Optimization (SMO) 8 3.1 The SMO Algorithm 8 3.2 Keerthis Improvement to SMO 9 3.3 Parallel SMO 10 3.3.1 Partitioning 10 3.3.2 Combination and Retraining 11 4 An Implementation of Parallel SMO using Cilk 12 5 An Implementation of Parallel SMO using Java Threads 14 5.1 Java Implementation details 14 5.2 Kernel Cache 15 5.3 Java Results 16 5.3.1 Results on Sunfire 17 6 Conclusion 19 Abstract Support Vector Machine (SVM) is an algorithmic technique for pattern classification that has grown in popularity in recent times, and has been used in many fields including bioinformatics. Several recent results provide improvements in various SVM algorithms. The Sequential Minimal Optimization (SMO) is one of the fastest and popular algorithms. This report describes parallelization of the SMO. The main idea is to partition the training data into more than one part, train these parts in parallel, and perform a combination and a retrain of the combined result. We have implemented this idea both in Cilk[13] as well as using Java threads. We also consider new techniques such as Keerthis improvement [8] to the SMO. Introduction Support Vector Machine (SVM) is an algorithm that was developed for pattern classification but has recently been adapted for other uses, such as finding regression and distribution estimation. It has been used in many fields such as bioinformatics, and is currently a very active research area in many universities and research institutes which include the National University of Singapore (NUS) and Massachusetts Institute of Technology (MIT). Since its introduction in 1970 by