Universal Approximation Theorem

Mathematical theorem stating that a neural network with a nonlinear activation function and enough neurons can approximate any continuous function to arbitrary precision.

1 articleLast mentioned

The Universal Approximation Theorem states that a feedforward neural network with even a single hidden layer can approximate any continuous function defined on a bounded domain to arbitrarily small error, provided it has a nonlinear activation function and enough neurons. George Cybenko proved it for sigmoid activations in 1989, and Kurt Hornik extended it to general nonlinear activation functions in 1991.

The theorem provides the theoretical basis for the claim that neural networks can learn complex relationships that linear models cannot capture. It is an existence result only: it says nothing about how many neurons are needed or whether gradient descent will actually find the required weights. Deep networks are used in practice because they achieve the same expressive power with far fewer parameters.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Pre-Training, Fine-Tuning and RL: Where Frontier LLMs Get Their QualityAI Performance ·

    The Universal Approximation Theorem says a network with non-linearities and enough parameters can approximate essentially any continuous function.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.