Extended Mixture of MLP Experts by Hybrid of Conjugate Gradient Method and Modified Cuckoo Search.pdfVIP

International Journal of Artificial Intelligence Applications (IJAIA), Vol.3, No.1, January 2012 Extended Mixture of MLP Experts by Hybrid of Conjugate Gradient Method and Modified Cuckoo Search 1 1,2 1 1 Hamid Salimi , Davar Giveki , Mohammad Ali Soltanshahi , Javad Hatami 1 School of Mathematics and Computer Science, University of Tehran, Tehran, Iran salimi.hamid86@, ali.soltanshahi@, jvdhtm@ 2 Department of Computer Science Universität des Saarlandes, Saarbrücken, Germany s9dagive@stud.uni-saarland.de Abstract This paper investigates a new method for improving the learning algorithm of Mixture of Experts (ME) model using a hybrid of Modified Cuckoo Search (MCS) and Conjugate Gradient (CG) as a second order optimization technique. The CG technique is combined with Back-Propagation (BP) algorithm to yield a much more efficient learning algorithm for ME structure. In addition, the experts and gating networks in enhanced model are replaced by CG based Multi-Layer Perceptrons (MLPs) to provide faster and more accurate learning. The CG is considerably depends on initial weights of connections of Artificial Neural Network (ANN), so, a metaheuristic algorithm, the so-called Modified Cuckoo Search is applied in order to select the optimal weights. The performance of proposed method is compared with Gradient Decent Based ME (GDME) and Conjugate Gradient Based ME (CGME) in classification and regression problems. The experimental results show that hybrid MSC and CG based ME (MCS-CGME) has faster convergence and better performance in utilized benchmark data sets. KEYWORDS Back Propagation (BP) algorithm, Gradient Decent (GD), Conjugate Gradient (CG), Modified Cuckoo Search (MCS), Mixture of Expert