no code implementations • 15 Nov 2023 • Vaishnavi Shrivastava, Percy Liang, Ananya Kumar
To maintain user trust, large language models (LLMs) should signal low confidence on examples where they are incorrect, instead of misleading the user.
1 code implementation • 8 Nov 2023 • Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, Tushar Khot
Our experiments with ChatGPT-3. 5 show that this bias is ubiquitous - 80% of our personas demonstrate bias; it is significant - some datasets show performance drops of 70%+; and can be especially harmful for certain groups - some personas suffer statistically significant drops on 80%+ of the datasets.
no code implementations • 3 Oct 2023 • Xiang Lisa Li, Vaishnavi Shrivastava, Siyan Li, Tatsunori Hashimoto, Percy Liang
To improve the consistency of LMs, we propose to finetune on the filtered generator and validator responses that are GV-consistent, and call this approach consistency fine-tuning.
no code implementations • 27 Nov 2021 • Vaishnavi Shrivastava, Radhika Gaonkar, Shashank Gupta, Abhishek Jha
Fine-tuning pre-trained language models improves the quality of commercial reply suggestion systems, but at the cost of unsustainable training times.
no code implementations • NAACL 2022 • FatemehSadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi, Taylor Berg-Kirkpatrick, Robert Sim, Dimitrios Dimitriadis
As such, these models are often unable to produce personalized responses for individual users, based on their data.