no code implementations • 9 Apr 2024 • Feng Liang, Zhen Zhang, Haifeng Lu, Victor C. M. Leung, Yanyi Guo, Xiping Hu
Due to intensive synchronization of models and sharing of data across GPUs and computing nodes during distributed training and inference processes, communication efficiency becomes the bottleneck for achieving high performance at a large scale.
1 code implementation • 25 Mar 2024 • Feiteng Fang, Liang Zhu, Min Yang, Xi Feng, Jinchang Hou, Qixuan Zhao, Chengming Li, Xiping Hu, Ruifeng Xu
Reinforcement learning from human feedback (RLHF) is a crucial technique in aligning large language models (LLMs) with human preferences, ensuring these LLMs behave in beneficial and comprehensible ways to users.
1 code implementation • 26 Feb 2024 • Shiwen Ni, Minghuan Tan, Yuelin Bai, Fuqiang Niu, Min Yang, BoWen Zhang, Ruifeng Xu, Xiaojun Chen, Chengming Li, Xiping Hu, Ye Li, Jianping Fan
In this paper, we contribute a new benchmark, the first Multilingual-oriented quiZ on Intellectual Property (MoZIP), for the evaluation of LLMs in the IP domain.
no code implementations • 26 Feb 2024 • Shiwen Ni, Min Yang, Ruifeng Xu, Chengming Li, Xiping Hu
To solve the inconsistency between training and inference caused by the randomness of dropout, some studies use consistency training to regularize dropout at the output layer.
no code implementations • 10 Feb 2024 • Zhibo Chu, Shiwen Ni, Zichong Wang, Xi Feng, Chengming Li, Xiping Hu, Ruifeng Xu, Min Yang, Wenbin Zhang
Language models serve as a cornerstone in natural language processing (NLP), utilizing mathematical methods to generalize language laws and knowledge for prediction and generation.
1 code implementation • 29 Jan 2024 • Jinchang Hou, Chang Ao, Haihong Wu, Xiangtao Kong, Zhigang Zheng, Daijia Tang, Chengming Li, Xiping Hu, Ruifeng Xu, Shiwen Ni, Min Yang
The integration of LLMs and education is getting closer and closer, however, there is currently no benchmark for evaluating LLMs that focuses on the Chinese K-12 education domain.
no code implementations • 14 Nov 2023 • Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, Min Yang
In this paper, we propose a new paradigm for fine-tuning called F-Learning (Forgetting before Learning), which employs parametric arithmetic to facilitate the forgetting of old knowledge and learning of new knowledge.
1 code implementation • 24 Oct 2023 • Jing Xiong, Chengming Li, Min Yang, Xiping Hu, Bin Hu
To this end, we design an Expression Syntax Information Bottleneck method for MWP (called ESIB) based on variational information bottleneck, which extracts essential features of expression syntax tree while filtering latent-specific redundancy containing syntax-irrelevant features.
no code implementations • 31 Mar 2023 • Yanjie Dong, Luya Wang, Yuanfang Chi, Jia Wang, Haijun Zhang, Fei Richard Yu, Victor C. M. Leung, Xiping Hu
A wireless federated learning system is investigated by allowing a server and workers to exchange uncoded information via orthogonal wireless channels.
no code implementations • 27 Oct 2022 • Jing Xiong, Zhongwei Wan, Xiping Hu, Min Yang, Chengming Li
Specifically, we firstly obtain a sub-network by pruning a roberta2tree model, for the sake to use the gap on output distribution between the original roberta2tree model and the pruned sub-network to expose spurious correlative samples.
no code implementations • 4 Jan 2022 • Jingjing Yang, Haifeng Lu, Chengming Li, Xiping Hu, Bin Hu
Gait analysis provides a non-contact, low-cost, and efficient early screening method for depression.
1 code implementation • 5 Jul 2021 • Haocong Rao, Xiping Hu, Jun Cheng, Bin Hu
In this paper, we for the first time propose a Self-supervised Multi-scale Skeleton Graph Encoding (SM-SGE) framework that comprehensively models human body, component relations, and skeleton dynamics from unlabeled skeleton graphs of various scales to learn an effective skeleton representation for person Re-ID.
no code implementations • 20 Jun 2021 • Yijiang Li, Wentian Cai, Ying Gao, Chengming Li, Xiping Hu
The local and detailed feature from the shallower layer such as boundary and tissue texture is particularly more important in medical segmentation compared with natural image segmentation.
1 code implementation • 6 Jun 2021 • Haocong Rao, Shihao Xu, Xiping Hu, Jun Cheng, Bin Hu
To fully explore body relations, we construct graphs to model human skeletons from different levels, and for the first time propose a Multi-level Graph encoding approach with Structural-Collaborative Relation learning (MG-SCR) to encode discriminative graph features for person Re-ID.
1 code implementation • 14 Nov 2020 • Shihao Xu, Haocong Rao, Xiping Hu, Bin Hu
Existing approaches usually learn action representations by sequential prediction but they suffer from the inability to fully learn semantic information.
1 code implementation • 5 Sep 2020 • Haocong Rao, Siqi Wang, Xiping Hu, Mingkui Tan, Yi Guo, Jun Cheng, Xinwang Liu, Bin Hu
This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID.
1 code implementation • 21 Aug 2020 • Haocong Rao, Siqi Wang, Xiping Hu, Mingkui Tan, Huang Da, Jun Cheng, Bin Hu
Unlike previous methods, we for the first time propose a generic gait encoding approach that can utilize unlabeled skeleton data to learn gait representations in a self-supervised manner.
2 code implementations • 1 Aug 2020 • Haocong Rao, Shihao Xu, Xiping Hu, Jun Cheng, Bin Hu
In this paper, we for the first time propose a contrastive action learning paradigm named AS-CAL that can leverage different augmentations of unlabeled skeleton data to learn action representations in an unsupervised manner.
no code implementations • 13 Mar 2020 • Shihao Xu, Jing Fang, Xiping Hu, Edith Ngai, Wei Wang, Yi Guo, Victor C. M. Leung
This article reviews current research on gait-based emotion detection, particularly on how gait parameters can be affected by different emotion states and how the emotion states can be recognized through distinct gait patterns.
no code implementations • 20 Feb 2020 • Hanshu Cai, Yiwen Gao, Shuting Sun, Na Li, Fuze Tian, Han Xiao, Jianxiu Li, Zhengwu Yang, Xiaowei Li, Qinglin Zhao, Zhenyu Liu, Zhijun Yao, Minqiang Yang, Hong Peng, Jing Zhu, Xiaowei Zhang, Guoping Gao, Fang Zheng, Rui Li, Zhihua Guo, Rong Ma, Jing Yang, Lan Zhang, Xiping Hu, Yumin Li, Bin Hu
The EEG dataset includes not only data collected using traditional 128-electrodes mounted elastic cap, but also a novel wearable 3-electrode EEG collector for pervasive applications.
no code implementations • 25 Sep 2019 • JieZhang Cao, Jincheng Li, Xiping Hu, Peilin Zhao, Mingkui Tan
ii) the $W$-distance of a specific layer to the target distribution tends to decrease along training iterations.
no code implementations • 22 Apr 2018 • Fusheng Hao, Jun Cheng, Lei Wang, Xinchao Wang, Jianzhong Cao, Xiping Hu, Dapeng Tao
Discriminative features are obtained by constraining the deep CNNs to map training samples to the corresponding anchors as close as possible.