no code implementations • EMNLP 2021 • Maharshi Gor, Kellie Webster, Jordan Boyd-Graber
The goal of question answering (QA) is to answer _any_ question.
1 code implementation • 15 Dec 2022 • Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie Webster
We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development.
no code implementations • 31 Oct 2022 • Reinald Kim Amplayo, Kellie Webster, Michael Collins, Dipanjan Das, Shashi Narayan
Large language models (LLMs) have been shown to perform well in answering questions and in producing long-form texts, both in few-shot closed-book settings.
no code implementations • NAACL (WOAH) 2022 • Zee Fryer, Vera Axelrod, Ben Packer, Alex Beutel, Jilin Chen, Kellie Webster
A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the input is changed?
no code implementations • 13 Dec 2021 • Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V Le, Yonghui Wu, Zhifeng Chen, Claire Cui
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing.
Ranked #10 on Language Modelling on LAMBADA
no code implementations • 15 Apr 2021 • Maharshi Gor, Kellie Webster, Jordan Boyd-Graber
The goal of question answering (QA) is to answer any question.
no code implementations • 7 Apr 2021 • Christian Hardmeier, Marta R. Costa-jussà, Kellie Webster, Will Radford, Su Lin Blodgett
At the Workshop on Gender Bias in NLP (GeBNLP), we'd like to encourage authors to give explicit consideration to the wider aspects of bias and its social implications.
no code implementations • 12 Feb 2021 • Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, Melvin Johnson
Responsible development of technology involves applications being inclusive of the diverse set of users they hope to support.
no code implementations • 6 Nov 2020 • Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne, Rajiv Raman, Kim Ramasamy, Rory Sayres, Jessica Schrouff, Martin Seneviratne, Shannon Sequeira, Harini Suresh, Victor Veitch, Max Vladymyrov, Xuezhi Wang, Kellie Webster, Steve Yadlowsky, Taedong Yun, Xiaohua Zhai, D. Sculley
Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains.
no code implementations • 12 Oct 2020 • Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, Slav Petrov
Pre-trained models have revolutionized natural language understanding.
2 code implementations • EMNLP 2020 • Ana Valeria Gonzalez, Maria Barrett, Rasmus Hvingelby, Kellie Webster, Anders Søgaard
The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are "hallucinatory", e. g., disambiguating gender-ambiguous occurrences of 'doctor' as male doctors.
no code implementations • 16 Jun 2020 • Kellie Webster, Emily Pitler
Machine translation systems with inadequate document understanding can make errors when translating dropped or neutral pronouns into languages with gendered pronouns (e. g., English).
no code implementations • ACL 2020 • Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, Stephen Denuyl
Building equitable and inclusive NLP technologies demands consideration of whether and how social attitudes are represented in ML models.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Hila Gonen, Kellie Webster
The successful application of neural methods to machine translation has realized huge quality advances for the community.
no code implementations • WS 2019 • Kellie Webster, Marta R. Costa-juss{\`a}, Christian Hardmeier, Will Radford
The 1st ACL workshop on Gender Bias in Natural Language Processing included a shared task on gendered ambiguous pronoun (GAP) resolution.
4 code implementations • TACL 2018 • Kellie Webster, Marta Recasens, Vera Axelrod, Jason Baldridge
Coreference resolution is an important task for natural language understanding, and the resolution of ambiguous pronouns a longstanding challenge.
no code implementations • EMNLP 2018 • Ali Elkahky, Kellie Webster, Daniel Andor, Emily Pitler
English part-of-speech taggers regularly make egregious errors related to noun-verb ambiguity, despite having achieved 97{\%}+ accuracy on the WSJ Penn Treebank since 2002.