Kernel regression and backpropagation training with noise

It is concluded that additive noise can improve the generalization capability of a feedforward network trained with the backpropagation approach, but the magnitude of the noise cannot be selected blindly, though.

Abstract

One method proposed for improving the generalization capability of a feedforward network trained with the backpropagation algorithm is to use artificial training vectors which are obtained by adding noise to the original training vectors. The authors discuss the connection of such backpropagation training with noise to kernel density and kernel regression estimation. They compare by simulated examples backpropagation, backpropagation with noise, and kernel regression in mapping estimation and pattern classification contexts. It is concluded that additive noise can improve the generalization capability of a feedforward network trained with the backpropagation approach. The magnitude of the noise cannot be selected blindly, though. Cross-validation-type procedures seem to be well suited for the selection of noise magnitude. Kernel regression, however, seems to perform well whenever backpropagation with noise performs well.>