1 code implementation • 3 Apr 2024 • Hussein Mozannar, Valerie Chen, Mohammed Alsobay, Subhro Das, Sebastian Zhao, Dennis Wei, Manish Nagireddy, Prasanna Sattigeri, Ameet Talwalkar, David Sontag
Evaluation of large language models (LLMs) for code has primarily relied on static benchmarks, including HumanEval (Chen et al., 2021), which measure the ability of LLMs to generate complete code that passes unit tests.
no code implementations • 3 Mar 2024 • Hyewon Jeong, Sarah Jabbour, Yuzhe Yang, Rahul Thapta, Hussein Mozannar, William Jongwon Han, Nikita Mehandru, Michael Wornow, Vladislav Lialin, Xin Liu, Alejandro Lozano, Jiacheng Zhu, Rafal Dariusz Kocielnik, Keith Harrigian, Haoran Zhang, Edward Lee, Milos Vukadinovic, Aparna Balagopalan, Vincent Jeanselme, Katherine Matton, Ilker Demirel, Jason Fries, Parisa Rashidi, Brett Beaulieu-Jones, Xuhai Orson Xu, Matthew McDermott, Tristan Naumann, Monica Agrawal, Marinka Zitnik, Berk Ustun, Edward Choi, Kristen Yeom, Gamze Gursoy, Marzyeh Ghassemi, Emma Pierson, George Chen, Sanjat Kanjilal, Michael Oberst, Linying Zhang, Harvineet Singh, Tom Hartvigsen, Helen Zhou, Chinasa T. Okolo
The organization of the research roundtables at the conference involved 17 Senior Chairs and 19 Junior Chairs across 11 tables.
no code implementations • 17 Jan 2024 • Niklas Mannhardt, Elizabeth Bondi-Kelly, Barbara Lam, Chloe O'Connell, Mercy Asiedu, Hussein Mozannar, Monica Agrawal, Alejandro Buendia, Tatiana Urman, Irbaz B. Riaz, Catherine E. Ricciardi, Marzyeh Ghassemi, David Sontag
Augmentations were evaluated for errors by clinicians, and we found misleading errors occur, with errors more common in real donated notes than synthetic notes, illustrating the importance of carefully written clinical notes.
1 code implementation • NeurIPS 2023 • Hussein Mozannar, Jimin J Lee, Dennis Wei, Prasanna Sattigeri, Subhro Das, David Sontag
In this work, we propose to learn rules, grounded in data regions and described in natural language, that illustrate how the human should collaborate with the AI.
1 code implementation • 8 Jun 2023 • Hussein Mozannar, Gagan Bansal, Adam Fourney, Eric Horvitz
Using data from 535 programmers, we perform a retrospective evaluation of CDHF and show that we can avoid displaying a significant fraction of suggestions that would have been rejected.
no code implementations • 26 May 2023 • Hussein Mozannar, Yuria Utsumi, Irene Y. Chen, Stephanie S. Gervasi, Michele Ewing, Aaron Smith-McLallen, David Sontag
This work presents the implementation of a real-world ML-based system to assist care managers in identifying pregnant patients at risk of complications.
1 code implementation • 15 Jan 2023 • Hussein Mozannar, Hunter Lang, Dennis Wei, Prasanna Sattigeri, Subhro Das, David Sontag
We show that prior approaches can fail to find a human-AI system with low misclassification error even when there exists a linear classifier and rejector that have zero error (the realizable setting).
1 code implementation • 25 Oct 2022 • Hussein Mozannar, Gagan Bansal, Adam Fourney, Eric Horvitz
However, to fully realize their potential, we must understand how programmers interact with these systems and identify ways to improve that interaction.
1 code implementation • 19 Jul 2022 • Mohammad-Amin Charusaie, Hussein Mozannar, David Sontag, Samira Samadi
One of the goals of learning algorithms is to complement and reduce the burden on human decision makers.
1 code implementation • 22 Nov 2021 • Hussein Mozannar, Arvind Satyanarayan, David Sontag
For this collaboration to perform properly, the human decision maker must have a mental model of when and when not to rely on the agent.
1 code implementation • ICML 2020 • Hussein Mozannar, David Sontag
Learning algorithms are often used in conjunction with expert decision makers in practical scenarios, however this fact is largely ignored when designing these algorithms.
1 code implementation • ICML 2020 • Hussein Mozannar, Mesrob I. Ohannessian, Nathan Srebro
Sensitive attributes such as race are rarely available to learners in real world settings as their collection is often restricted by laws and regulations.
1 code implementation • WS 2019 • Hussein Mozannar, Karl El Hajal, Elie Maamary, Hazem Hajj
Our system for open domain question answering in Arabic (SOQAL) is based on two components: (1) a document retriever using a hierarchical TF-IDF approach and (2) a neural reading comprehension model using the pre-trained bi-directional transformer BERT.
no code implementations • 7 Dec 2018 • Hussein Mozannar, Mesrob I. Ohannessian, Nathan Srebro
In this paper, we propose a simple yet revealing model that encompasses (1) a selection process where an institution chooses from multiple groups according to their qualifications so as to maximize an institutional utility and (2) dynamics that govern the evolution of the groups' qualifications according to the imposed policies.