no code implementations • 8 Nov 2023 • Timm Hess, Tinne Tuytelaars, Gido M. van de Ven
Recent years have seen considerable progress in the continual training of deep neural networks, predominantly thanks to approaches that add replay or regularization terms to the loss function to approximate the joint loss over all tasks so far.
no code implementations • 3 Apr 2023 • Timm Hess, Eli Verwimp, Gido M. van de Ven, Tinne Tuytelaars
Carefully taking both aspects into account, we show that, even though it is true that feature forgetting can be small in absolute terms, newly learned information tends to be forgotten just as catastrophically at the level of the representation as it is at the output level.
2 code implementations • 4 Jun 2021 • Timm Hess, Martin Mundt, Iuliia Pliushch, Visvanathan Ramesh
Several families of continual learning techniques have been proposed to alleviate catastrophic interference in deep neural network training on non-stationary data.