Multilayer feedforward networks with a nonpolynomial activation function can approximate any function

It is shown that a standard multilayer feedforward network with a locally bounded piecewise continuous activation function can approximate any continuous function to any degree of accuracy if and only if the network's activation function is not a polynomial.

Abstract

Several researchers characterized the activation function under which multilayer feedforward\nnetworks can act as universal approximators. We show that most of all the characterizations\nthat were reported thus far in the literature are special cases of the following\ngeneral result: a standard multilayer feedforward network with a locally bounded piecewise\ncontinuous activation function can approximate any continuous function to any degree of\naccuracy if and only if the network's activation function is not a polynomial. We also\nemphasize the important role of the threshold, asserting that without it the last theorem\ndoes not hold.