How many neurons does it take to approximate any function?

Neural networks are mathematically guaranteed to approximate any continuous function well enough, but the hard question is how many neurons you actually need. This survey traces four decades of theory on approximation rates, showing how deeper networks dramatically improve parameter efficiency compared to shallow ones, and reviews recent work on Kolmogorov-Arnold networks as an alternative architecture. Understanding these tradeoffs matters for practitioners deciding how to structure networks for efficient learning.