Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Data

no code implementations • 23 May 2024 • Haoran Li, Xinyuan Zhao, Dadi Guo, Hanlin Gu, Ziqian Zeng, Yuxing Han, Yangqiu Song, Lixin Fan, Qiang Yang

In this paper, we introduce a Federated Domain-specific Knowledge Transfer (FDKT) framework, which enables domain-specific knowledge transfer from LLMs to SLMs while preserving clients' data privacy.

Federated Learning Transfer Learning

PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models

1 code implementation • 7 Nov 2023 • Haoran Li, Dadi Guo, Donghao Li, Wei Fan, Qi Hu, Xin Liu, Chunkit Chan, Duanyi Yao, Yuan YAO, Yangqiu Song

Lastly, PrivLM-Bench performs existing privacy attacks on LMs with pre-defined privacy objectives as the empirical evaluation results.

Privacy Preserving

Multi-step Jailbreaking Privacy Attacks on ChatGPT

1 code implementation • 11 Apr 2023 • Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, Yangqiu Song

With the rapid progress of large language models (LLMs), many downstream NLP tasks can be well solved given appropriate prompts.