no code implementations • 27 May 2024 • Biqing Qi, Junqi Gao, Kaiyan Zhang, Dong Li, Jianxing Liu, Ligang Wu, BoWen Zhou
Experiments on long-range modeling tasks in autoregressive language modeling and Long Range Arena demonstrate the general effectiveness of the SMR mechanism for a series of SSM models.
no code implementations • 7 Mar 2024 • Biqing Qi, Junqi Gao, Xingquan Chen, Dong Li, Jianxing Liu, Ligang Wu, BoWen Zhou
However, current EM-based methods retrieves memory globally by performing Vector-to-Vector (V2V) interaction between features corresponding to the input and prototypes stored in EM, neglecting the geometric structure of local features.
1 code implementation • 5 Mar 2024 • Biqing Qi, Xingquan Chen, Junqi Gao, Dong Li, Jianxing Liu, Ligang Wu, BoWen Zhou
Drawing on Complementary Learning System theory, this paper presents a novel Interactive Continual Learning (ICL) framework, enabled by collaborative interactions among models of various sizes.
no code implementations • 26 Feb 2024 • Biqing Qi, Junqi Gao, Yiang Luo, Jianxing Liu, Ligang Wu, BoWen Zhou
The rise of generative neural networks has triggered an increased demand for intellectual property (IP) protection in generated content.