Efficient Deep Approximation of GMMs

no code implementations • NeurIPS 2019 • Shirin Jalali, Carl Nuzman, Iraj Saniee

The universal approximation theorem states that any regular function can be approximated closely using a single hidden layer neural network.

General Classification

Efficient Deep Learning of GMMs

no code implementations • 15 Feb 2019 • Shirin Jalali, Carl Nuzman, Iraj Saniee

We show that a collection of Gaussian mixture models (GMMs) in $R^{n}$ can be optimally classified using $O(n)$ neurons in a neural network with two hidden layers (deep neural network), whereas in contrast, a neural network with a single hidden layer (shallow neural network) would require at least $O(\exp(n))$ neurons or possibly exponentially large coefficients.

General Classification