Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions

no code implementations • 13 Dec 2021 • Martin Hutzenthaler, Arnulf Jentzen, Katharina Pohl, Adrian Riekert, Luca Scarpa

In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day it remains an open problem of research to provide a mathematical convergence analysis which rigorously explains the success of SGD type optimization methods in the training of DNNs.

An extended variational theory for nonlinear evolution equations via modular spaces

no code implementations • 10 Dec 2020 • Alexander Menovschikov, Anastasia Molchanova, Luca Scarpa

We propose an extension of the classical variational theory of evolution equations that accounts for dynamics also in possibly non-reflexive and non-separable spaces.

Analysis of PDEs Functional Analysis 35A15, 35D30, 35K67