DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation

1 code implementation • 27 Feb 2024 • Sunghyeon Woo, Baeseong Park, Byeongwook Kim, Minjung Jo, Sejung Kwon, Dongsuk Jeon, Dongsoo Lee

In this paper, we propose Dropping Backward Propagation (DropBP), a novel approach designed to reduce computational costs while maintaining accuracy.