no code implementations • 27 May 2024 • Robert Wolfe, Isaac Slaughter, Bin Han, Bingbing Wen, Yiwei Yang, Lucas Rosenblatt, Bernease Herman, Eva Brown, Zening Qu, Nic Weber, Bill Howe
The rapid proliferation of generative AI has raised questions about the competitiveness of lower-parameter, locally tunable, open-weight models relative to high-parameter, API-guarded, closed-weight models in terms of performance, domain adaptation, cost, and generalization.
no code implementations • 24 May 2024 • Robert Wolfe, Tanushree Mitra
Generative AI appears poised to transform white collar professions, with more than 90% of Fortune 500 companies using OpenAI's flagship GPT models, which have been characterized as "general purpose technologies" capable of effecting epochal changes in the economy.
1 code implementation • 7 Jul 2023 • Shiva Omrani Sabbaghi, Robert Wolfe, Aylin Caliskan
Adapting the projection-based approach to embedding association tests that quantify bias, we find that language models exhibit the most biased attitudes against gender identity, social class, and sexual orientation signals in language.
1 code implementation • 21 Dec 2022 • Robert Wolfe, Yiwei Yang, Bill Howe, Aylin Caliskan
A first experiment uses standardized images of women from the Sexual OBjectification and EMotion Database, and finds that human characteristics are disassociated from images of objectified women: the model's recognition of emotional state is mediated by whether the subject is fully or partially clothed.
no code implementations • 1 Jul 2022 • Robert Wolfe, Aylin Caliskan
In an image captioning task, BLIP remarks upon the race of Asian individuals as much as 36% of the time, but never remarks upon race for White individuals.
no code implementations • 7 Jun 2022 • Aylin Caliskan, Pimparkar Parth Ajay, Tessa Charlesworth, Robert Wolfe, Mahzarin R. Banaji
Using the Single-Category Word Embedding Association Test, we demonstrate the widespread prevalence of gender biases that also show differences in: (1) frequencies of words associated with men versus women; (b) part-of-speech tags in gender-associated words; (c) semantic categories in gender-associated words; and (d) valence, arousal, and dominance in gender-associated words.
1 code implementation • 23 May 2022 • Robert Wolfe, Aylin Caliskan
The model is more likely to rank the unmarked "person" label higher than labels denoting gender for Male individuals (26. 7% of the time) vs.
1 code implementation • 22 May 2022 • Robert Wolfe, Mahzarin R. Banaji, Aylin Caliskan
We examine the state-of-the-art multimodal "visual semantic" model CLIP ("Contrastive Language Image Pretraining") for the rule of hypodescent, or one-drop rule, whereby multiracial people are more likely to be assigned a racial or ethnic label corresponding to a minority or disadvantaged racial or ethnic group than to the equivalent majority or advantaged group.
no code implementations • ACL 2022 • Robert Wolfe, Aylin Caliskan
We find that contrastive visual semantic pretraining significantly mitigates the anisotropy found in contextualized word embeddings from GPT-2, such that the intra-layer self-similarity (mean pairwise cosine similarity) of CLIP word embeddings is under . 25 in all layers, compared to greater than . 95 in the top layer of GPT-2.
1 code implementation • 14 Mar 2022 • Robert Wolfe, Aylin Caliskan
VAST, the Valence-Assessing Semantics Test, is a novel intrinsic evaluation task for contextualized word embeddings (CWEs).
no code implementations • EMNLP 2021 • Robert Wolfe, Aylin Caliskan
Moreover, we find Spearman's r between racial bias and name frequency in BERT of . 492, indicating that lower-frequency minority group names are more associated with unpleasantness.