Jan Larsen, Lars K. Hansen, Claes Svarer and Mattias Ohlsson Design and regularization of neural networks: the optimal use of avalidation set In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing VI, 62-71 (1996) Abstract: We derive novel algorithms for estimation of regularization parameters and for optimization of neural net architectures based on a validation set. Regularisation parameters are estimated using an iterative gradient descent scheme. Architecture optimization is performed by approximative combinatorial search among the relevant subsets of an initial neural network architecture by employing a validation set based optimal brain damage/surgeon (OBD/OBS) or a mean field combinatorial optimization approach. Numerical results with linear models and feed-forward neural networks demonstrate the viability of the methods.