Search Results for author:
Found 3 papers, 1 papers with code
Base of RoPE Bounds Context Length
We revisit the role of RoPE in LLMs and propose a novel property of long-term decay, we derive that the \textit{base of RoPE bounds context length}: there is an absolute lower bound for the base value to obtain certain context length capability.
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
As Large Language Models (LLMs) continue to advance in performance, their size has escalated significantly, with current LLMs containing billions or even trillions of parameters.
MDIA: A Benchmark for Multilingual Dialogue Generation in 46 Languages
In this paper, we present mDIA, the first large-scale multilingual benchmark for dialogue generation across low- to high-resource languages.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
