Accelerating SGD for Distributed Deep-Learning Using Approximated Hessian Matrix

no code implementations • 15 Sep 2017 • Sébastien M. R. Arnold, Chunming Wang

We introduce a novel method to compute a rank $m$ approximation of the inverse of the Hessian matrix in the distributed regime.

Second-order methods Stochastic Optimization