Mixture of Experts of ANN and KNN on the problem of Puzzle 8.pdfVIP

Mixture of Experts of ANN and KNN on the problem of Puzzle 8 Guiwen Hou Jingyue Zhang Jiahong Zhou Computing Science Department University of Alberta Edmonton, Candada , T6G 2E8 {guiwen,jingye,jiahong}@cs.ualberta.ca November 14,2002 Abstract: In this project we present a heuristic learning process by training ANN (artificial neural network) and KNN (k-nearest neighbor) using the best number of steps, gained from A*, from randomly generated states to the goal. After training ANN and KNN, the mixture of Experts is discussed and the empirical data are collected to demonstrate the feasibility and accuracy of combination of ANN and KNN in different areas. In our project, we present a novel method to combination of ANN and KNN by using the confidences. In order to achieve this goal, different approaches are adopted to converge the ANN. Key words: ANN (Artificial Neural Network), KNN (k-nearest neighbor), Auto-encoding ANN Contents 1.Problem formulation and Challenges 3 2.Related work 4 2.1 8/15puzzle search4 2.2 Artificial Neural Network and its confidence5 2.3 Function Approximation5 2.4 KNN and its confidence 5 2.5 Mixture of experts6 3. Combination of ANN and KNN (Noveld Methods)6 3.1 Generate training samples by A*: 6 3.2 Build ANN and train ANN 7 3.3 Scaling 9 3.4 Momentum9 3.5 Learning rate10 3.6 Using Auto-encoding network to calculate the confidence of ANN10 3.7 Ensemble of auto-encoding ANNs 12 3.8 Training KNN13 4 Empirical result and analysis15 4.1 Converge of ANN 15 4.2 Combine ANN and KNN 17 4.3 Apply this method to other domain18 5 Summary 21 5.1 Conclusion21 5.2 Discussion22 5.3 Future work22 Acknowledgement 23 Bibliography24 1.Problem formulation and Challenges Admissible heuristics grantee the optimal path solutions in search algorithms such as A*, but typically the more accurate and effective the heuristic is, the more expensive the computation will be, especially