深度学习模型各层参数数目对于性能的影响.pdfVIP

Computer Science and Application 计算机科学与应用, 2015, 5(12), 445-453 Published Online December 2015 in Hans. /journal/csa /10.12677/csa.2015.512056 The Influence of the Amount of Parameters in Different Layers on the Performance of Deep Learning Models 1 2 1* Xibin Yue , Xiaolin Hu , Liang Tang 1 College of Engineering, Beijing Forestry University, Beijing 2 Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing th th th Received: Dec. 10 , 2015; accepted: Dec. 27 , 2015; published: Dec. 30 , 2015 Copyright © 2015 by authors and Hans Publishers Inc. This work is licensed under the Creative Commons Attribution International License (CC BY). /licenses/by/4.0/ Abstract In recent years, deep learning has been widely used in many pattern recognition tasks including im- age classification and speech recognition due to its excellent performance. But a general rule for the structure design is lacked. We explored the influence of the amount of parameters in different layers of two deep learning models, convolutional neural network (CNN) and recurrent convolutional neu- ral network (RCNN). Experiments on three benchmark datasets, CIFAR-10, CIFAR-100 and SVHN showed that when the total number of parameters was fixed, increasing the number of parameters in higher layers could boost the performance of the models while increasing the number of parame- ters in lower layers could be harmful to the performance of the models. Based on this simple rule, we obtained the state-of-the-art classification accuracy on CIFAR-100 and SVHN with single models.