Analysis of Tikhonov regularization for function approximation by neural networks.pdfVIP

Analysis of Tikhonov Regularization for Function Approximation by Neural Networks Martin Burger and Andreas Neubauer Institut f ur Industriemathematik JohannesKeplerUniversitat A Linz Austria Abstract This pap er is devoted to the convergence and stability analysis of Tikhonov regularization for function approximation by a class of feedforward neural networks with one hidden layer and linear output layer We investigate two frequently used approaches namely regularization by output smoothing and regularization by weight decay as well as a combination of b oth metho ds to combine their advantages We show that in all cases stable approximations are obtained converging to the approximated function in a desired Sob olev space as the noise in the data tends to zero in the weaker 2 L norm if the regularization parameter and the numb er of units in the network are chosen appropriately Under additional smo othness assumptions we are able to show convergence rates results in terms of the noise level and the numb er of units in the network In addition we show how the theoretical results can b e applied to the imp ortant classes of p erceptrons with one hidden layer and to translation networks Finally the p erformance of the dierent approaches is compared in some numerical examples Key Words Illp osed problems neural networks Tikhonov regularization output smo othing weight decay function approximation AMS Sub ject Classications J B A Intro duction m In this pap er we deal with the problem of approximating a function f H for 2 which only noisy measurements f L with an error b ound