Neural Networks Nonlinear Optimization for Constrained 神经网络的非线性优化的约束.docVIP

Neural Networks: Nonlinear Optimization for Constrained Learning and its Applications STAVROS J. PERANTONIS Computational Intelligence Laboratory, Institute of Informatics and Telecommunications, National Center for Scientific Research “Demokritos”, 153 10 Aghia Paraskevi, Athens, GREECE Abstract: - Feedforward neural networks are highly non-linear systems whose training is usually carried out by performing optimization of a suitable cost function which is nonlinear with respect to the optimization variables (synaptic weights). This paper summarizes the considerable advantages that arise from using constrained optimization methods for training feedforward neural networks. A constrained optimization framework is presented that allows incorporation of suitable constraints in the learning process. This enables us to include additional knowledge in efficient learning algorithms that can be either general purpose or problem specific. We present general purpose first order and second order algorithms that arise from the proposed constrained learning framework and evaluate their performance on several benchmark classification problems. Regarding problem specific algorithms, we present recent developments concerning the numerical factorization and root identification of polynomials using sigma-pi feedforward networks trained by constrained optimization techniques. Key-Words: - neural networks, constrained optimization, learning algorithms 1 Introduction Artificial neural networks are computational systems loosely modeled after the human brain. Although it is not yet clearly understood exactly how the brain optimizes its connectivity to perform perception and decision making tasks, there have been considerable successes in developing artificial neural network training algorithms by drawing from the wealth of methods available from the well established field of non-linear optimization. Although most of the methods used for supervised learning, including the original and