1 code implementation • 27 Feb 2024 • Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, Lingpeng Kong
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
no code implementations • 17 Oct 2023 • Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han, Pengcheng He
In this paper, we seek to empirically investigate knowledge transfer from larger to smaller models through a parametric perspective.
1 code implementation • 8 Oct 2023 • Xiaoran Liu, Hang Yan, Shuo Zhang, Chenxin An, Xipeng Qiu, Dahua Lin
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest.
3 code implementations • 20 Jul 2023 • Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, Xipeng Qiu
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories.
no code implementations • 23 May 2023 • Chenxin An, Jiangtao Feng, Fei Huang, Xipeng Qiu, Lingpeng Kong
In this paper, we propose to ease the difficulty of modality learning via sampling from the model distribution instead of the data distribution.
1 code implementation • COLING 2022 • Chenxin An, Ming Zhong, Zhiyong Wu, Qin Zhu, Xuanjing Huang, Xipeng Qiu
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives.
2 code implementations • 29 May 2022 • Chenxin An, Jiangtao Feng, Kai Lv, Lingpeng Kong, Xipeng Qiu, Xuanjing Huang
We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.
no code implementations • 20 Feb 2022 • Yitao Liu, Chenxin An, Xipeng Qiu
With the success of large-scale pre-trained models (PTMs), how efficiently adapting PTMs to downstream tasks has attracted tremendous attention, especially for PTMs with billions of parameters.
no code implementations • 18 Feb 2022 • Zhichao Geng, Hang Yan, Zhangyue Yin, Chenxin An, Xipeng Qiu
Chinese NER is a difficult undertaking due to the ambiguity of Chinese characters and the absence of word boundaries.
no code implementations • 16 Sep 2021 • Chenxin An, Ming Zhong, Zhichao Geng, Jianqiang Yang, Xipeng Qiu
Existing summarization systems mostly generate summaries purely relying on the content of the source document.
1 code implementation • 7 Apr 2021 • Chenxin An, Ming Zhong, Yiran Chen, Danqing Wang, Xipeng Qiu, Xuanjing Huang
Previous work for text summarization in scientific domain mainly focused on the content of the input document, but seldom considering its citation network.