MathematicsofControl,.PDF

Math. Control Signals Systems (1989) 2:303-314 Mathematics of Control, Signals, and Systems 1989 Springer-Verlag New York Inc. Approximation by Superpositions of a Sigmoidal Function* G. Cybenkot Abstr,,ct. In this paper we demonstrate that finite linear combinations of com- positions of a fixed, univariate function and a set ofaffine functionals can uniformly approximate any continuous function of n real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single bidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks. Key words. Neural networks, Approximation, Completeness. 1. Introduction A number of diverse application areas are concerned with the representation of general functions of an n-dimensional real variable, x R, by finite linear combina- tions of the form N + or), (1) j=l where yj R and ctj, 0 I~ are fixed. (yr is the transpose ofy so that yrx is the inner product of y and x.) Here the univariate function tr depends heavily on the context of the application. Our major concern is with so-called sigmoidal as: