no code implementations • 23 Feb 2024 • Yanjun Zhao, Sizhe Dang, Haishan Ye, Guang Dai, Yi Qian, Ivor W. Tsang
Fine-tuning large language models (LLMs) with classic first-order optimizers entails prohibitive GPU memory due to the backpropagation process.
no code implementations • 8 Feb 2024 • Yanjun Zhao, Tian Zhou, Chao Chen, Liang Sun, Yi Qian, Rong Jin
Time series analysis is vital for numerous applications, and transformers have become increasingly prominent in this domain.
Computational Efficiency Multivariate Time Series Forecasting +2
1 code implementation • 14 Jun 2023 • Yanjun Zhao, Ziqing Ma, Tian Zhou, Liang Sun, Mengni Ye, Yi Qian
On the other hand, the long input sequence usually leads to large model size and high time complexity.
no code implementations • 3 Aug 2022 • Xiang Ma, Haijian Sun, Rose Qingyang Hu, Yi Qian
Nevertheless, since it is the model instead of the raw data that is shared, the system can be exposed to the poisoning model attacks launched by malicious clients.
1 code implementation • 2 Jul 2022 • Zhongyuan Zhang, Yi Qian, Yanxiang Zhao, Lin Zhu, Jinjin Wang
In this paper, the noise image extracted by the improved constrained convolution is used as the input of the model instead of the original image to obtain more subtle traces of manipulation.
no code implementations • 1 Jan 2021 • Lujun Li, Yikai Wang, Anbang Yao, Yi Qian, Xiao Zhou, Ke He
In this paper, we present Explicit Connection Distillation (ECD), a new KD framework, which addresses the knowledge distillation problem in a novel perspective of bridging dense intermediate feature connections between a student network and its corresponding teacher generated automatically in the training, achieving knowledge transfer goal via direct cross-network layer-to-layer gradients propagation, without need to define complex distillation losses and assume a pre-trained teacher model to be available.
no code implementations • 14 Aug 2020 • Shilin Xu, Caili Guo, Rose Qingyang Hu, Yi Qian
To support the ever increasing computational needs in such a vehicular network, the distributed virtual cloud network (VCN) is formed, based on which a computational resource sharing scheme through offloading among nearby vehicles is proposed.