Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators
A new deep neural network called DeepONet can lean various mathematical operators with small generalization error and can learn various explicit operators, such as integrals and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations.
Abstract
While it is widely known that neural networks are universal approximators of\ncontinuous functions, a less known and perhaps more powerful result is that a\nneural network with a single hidden layer can approximate accurately any\nnonlinear continuous operator. This universal approximation theorem is\nsuggestive of the potential application of neural networks in learning\nnonlinear operators from data. However, the theorem guarantees only a small\napproximation error for a sufficient large network, and does not consider the\nimportant optimization and generalization errors. To realize this theorem in\npractice, we propose deep operator networks (DeepONets) to learn operators\naccurately and efficiently from a relatively small dataset. A DeepONet consists\nof two sub-networks, one for encoding the input function at a fixed number of\nsensors $x_i, i=1,\\dots,m$ (branch net), and another for encoding the locations\nfor the output functions (trunk net). We perform systematic simulations for\nidentifying two types of operators, i.e., dynamic systems and partial\ndifferential equations, and demonstrate that DeepONet significantly reduces the\ngeneralization error compared to the fully-connected networks. We also derive\ntheoretically the dependence of the approximation error in terms of the number\nof sensors (where the input function is defined) as well as the input function\ntype, and we verify the theorem with computational results. More importantly,\nwe observe high-order error convergence in our computational tests, namely\npolynomial rates (from half order to fourth order) and even exponential\nconvergence with respect to the training dataset size.\n