OR-Bench: An Over-Refusal Benchmark for Large Language Models

no code implementations • 31 May 2024 • Justin Cui, Wei-Lin Chiang, Ion Stoica, Cho-Jui Hsieh

OR-Bench comprises 80, 000 seemingly toxic prompts across 10 common rejection categories, a subset of around 1, 000 hard prompts that are challenging even for state-of-the-art LLMs, and an additional 600 toxic prompts to prevent indiscriminate responses.

Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory

2 code implementations • 19 Nov 2022 • Justin Cui, Ruochen Wang, Si Si, Cho-Jui Hsieh

The resulting algorithm sets new SOTA on ImageNet-1K: we can scale up to 50 IPCs (Image Per Class) on ImageNet-1K on a single GPU (all previous methods can only scale to 2 IPCs on ImageNet-1K), leading to the best accuracy (only 5. 9% accuracy drop against full dataset training) while utilizing only 4. 2% of the number of data points - an 18. 2% absolute gain over prior SOTA.

DC-BENCH: Dataset Condensation Benchmark

2 code implementations • 20 Jul 2022 • Justin Cui, Ruochen Wang, Si Si, Cho-Jui Hsieh

Dataset Condensation is a newly emerging technique aiming at learning a tiny dataset that captures the rich information encoded in the original dataset.

Data Augmentation Data Compression +2