1 code implementation • 30 Mar 2024 • Aru Maekawa, Satoshi Kosugi, Kotaro Funakoshi, Manabu Okumura
To address this issue, we propose a novel text dataset distillation approach, called Distilling dataset into Language Model (DiLM), which trains a language model to generate informative synthetic training samples as text data, instead of directly optimizing synthetic samples.
1 code implementation • 8 Mar 2024 • Aru Maekawa, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura
Recently, decoder-only pre-trained large language models (LLMs), with several tens of billion parameters, have significantly impacted a wide range of natural language processing (NLP) tasks.