基于注意力模型的混合学习算法 - 软件学报.pdf

1000-9825/2005/16(06)1073 ©2005 Journal of Software 软 件 学 报 Vol.16, No.6 ∗ 基于注意力模型的混合学习算法 + 杨 博 , 苏小红, 王亚东 (哈尔滨工业大学 计算机科学与技术学院,黑龙江 哈尔滨 150001) A Hybrid Algorithm Based on Attention Model + YANG Bo , SU Xiao-Hong, WANG Ya-Dong (College of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China) + Corresponding author: Phn: +86-451 E-mail: boyang621@ 126.com, Received 2003-11-04; Accepted 2005-01-06 Yang B, Su XH, Wang YD. A hybrid algorithm based on attention model. Journal of Software, 2005,16(6): 1073−1080. DOI: 10.1360/jos161073 Abstract: A hybrid algorithm based on attention model (HAAM) is proposed to speed up the training of back-propagation neural networks and improve the performances. The algorithm combines the genetic algorithm with the BP algorithm based on magnified error signal. The key to this algorithm lies in the partition of the BP training process into many chips with each chip trained by the BP algorithm. The chips in the same iteration are optimized by the GA operators, and those in different iterations constitute the whole training. Therefore, the HAAM obtains the ability of searching the global optimum solution relying on these operations, and it is easy to be parallelly processed. The simulation experiments show that this algorithm can effectively avoid failure training caused by randomizing the initial weights and thresholds, and solve the slow convergence problem resulted from the Flat-Spots when the error signal becomes too small. Moreover, this algorithm improves the generalization of BP network by improving the training precision instead of adding hidden neurons. Key words: