DEPTH: Discourse Education through Pre-Training Hierarchically

1 code implementation • 13 May 2024 • Zachary Bamberger, Ofek Glick, Chaim Baskin, Yonatan Belinkov

Language Models (LMs) often struggle with linguistic understanding at the discourse level, even though discourse patterns such as coherence, cohesion, and narrative flow are prevalent in their pre-training data.

Decoder Natural Language Understanding +1