State-Space Systems as Dynamic Generative Models

no code implementations • 12 Apr 2024 • Juan-Pablo Ortega, Florian Rossmannek

The results in this paper constitute a significant stochastic generalization of sufficient conditions for the deterministic echo state property to hold, in the sense that the stochastic echo state property can be satisfied under contractivity conditions that are strictly weaker than those in deterministic situations.

Gradient descent provably escapes saddle points in the training of shallow ReLU networks

no code implementations • 3 Aug 2022 • Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek

Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms avoid so-called strict saddle points of the loss function.

Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions

no code implementations • 19 Mar 2021 • Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek

In this paper, we analyze the landscape of the true loss of neural networks with one hidden layer and ReLU, leaky ReLU, or quadratic activation.

A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions

no code implementations • 19 Feb 2021 • Patrick Cheridito, Arnulf Jentzen, Adrian Riekert, Florian Rossmannek

This Lyapunov function is the central tool in our convergence proof of the gradient descent method.

Non-convergence of stochastic gradient descent in the training of deep neural networks

no code implementations • 12 Jun 2020 • Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek

Deep neural networks have successfully been trained in various application areas with stochastic gradient descent.

Efficient approximation of high-dimensional functions with neural networks

no code implementations • 9 Dec 2019 • Patrick Cheridito, Arnulf Jentzen, Florian Rossmannek

In this paper, we develop a framework for showing that neural networks can overcome the curse of dimensionality in different high-dimensional approximation problems.

Numerical Analysis Numerical Analysis 68T07 I.2.0