Evaluating the Social Impact of Generative AI Systems in Systems and Society..Irene Solaiman*, Zeerak Talat*, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Canyu Chen, Hal Daumé III, Jesse Dodge, Isabella Duan, Felix Friedrich, Avijit Ghosh, Usman Gohar, Sara Hooker, Yacine Jernite, Ria Kalluri, Alberto Lusoli, Alina Leidinger, Michelle Lin, Xiuzhu Lin, Sasha Luccioni, Jennifer Mickel, Margaret Mitchell, Jessica Newman, Anaelia Ovalle, Marie-Therese Png, Shubham Singh, Andrew Strait, Lukas Struppek, Arjun Subramonian. Forthcoming Handbook of Generative AI. Oxford University Press.
[Book Chapter]
Content Moderation..Zeerak Talat. Forthcoming In Review.
[Book Chapter]
Detecting "Dirt" and "Toxicity": Rethinking Content Moderation as Pollution Behaviour..Nanna Bonde Thylstrup, Zeerak Talat. Forthcoming First Monday. First Monday.
[Journal Paper]
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models..Margaret Mitchell, Hamdan Al-Ali, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Kaustubh Dhole, Timm Dill, Amirbek Djanibekov, Tair Djanibekov, Jad Doughman, Ritam Dutt, Jessica Zosa Forde, Jay Gala, Avijit Ghosh, Sil Hamilton, Carolin Holtermann, Jerry Huang, Lucie-Aimée Kaffee, Janavi Kasera, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Sagnik Mukherjee, Nikita Nangia, Shangrui Nie, Anaelia Ovalle, Giada Pistilli, Esther Ploeger, Jeremy Qin, Dragomir Radev, Vipul Raheja, Beatrice Savoldi, Shanya Sharma, Xudong Shen, Karolina Stanczak, Arjun Subramonian, Kaiser Sun, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Emilio Villa Cueva, Marcelo Viridiano, Oskar van der Wal, Adina Yakefu, Kayo Yin, Mike Zhang, Sydney Zink, Aurélie Névéol, Zeerak Talat. Forthcoming In Review.
[Conference Paper]
Exploring the Limitations of Detecting Machine-Generated Text..Jad Doughman, Osama Mohammed Afsal, Hawau Olamine Toyin, Shady Shehata, Preslav Nakov, Zeerak Talat. Forthcoming In Review.
[Conference Paper]
2024
Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models Eddie L. Ungless, Nikolas Vitsakis, Zeerak Talat, James Garforth, Björn Ross, Arno Onken, Atoosa Kasirzadeh, Alexandra Birch 2024.
[Permaprint]
Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024). 2024. Proceedings of the 8th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Proceedings]
LLMs produce racist output when prompted in African American English Su Lin Blodgett, Zeerak Talat 2024. Nature
[Public Dissemination]
Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP Pieter Delebolle, Giuseppe Attanasio, Debora Nozza, Su Lin Blodgett, Zeerak Talat. 2024. The Proceedings of the 2024 Conference on Empircal Methods in Natural Language Processing. Association of Computational Linguistics.
[Conference Paper]
Understanding" Democratization" in NLP and ML Research. Arjun Subramonian, Vagrant Gautam, Dietrich Klakow, Zeerak Talat. 2024. The Proceedings of the 2024 Conference on Empircal Methods in Natural Language Processing. Association of Computational Linguistics.
[Conference Paper]
Classist Tools: Social Class Correlates with Performance in NLP. Amanda Cercas Curry, Giuseppe Attanasio, Zeerak Talat, Dirk Hovy. 2024. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Association of Computational Linguistics.
[Conference Paper]
Documenting Geographically and Contextually Diverse Language Data Sources. Angelina McMillan-Major, Francesco De Toni, Zaid Alyafeai, Stella Biderman, Kimbo Chen, Gérard Dupont, Hady Elsahar, Chris Emezue, Alham Fikri Aji, Suzana Ilić, Nurulaqilla Khamis, Colin Leong, Maraim Masoud, Aitor Soroa, Pedro Ortiz Suarez, Daniel van Strien, Zeerak Talat, Yacine Jernite 2024. Northern European Journal of Language Technology.
[Journal Paper]
ARAOFFENSE: Detecting Offensive Speech Across Dialects in Arabic Media Youssef Nafea, Shady Shehata, Zeerak Talat, Ahmed Aboeitta, Ahmed Sharshar, Preslav Nakov. 2024. Proceedings of Interspeech 2024. ISCA
[Conference Paper]
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels Eve Fleisig, Su Lin Blodgett, Dan Klein, Zeerak Talat. 2024. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) Association of Computational Linguistics.
[Conference Paper]
Impoverished Language Technology: The Lack of (Social) Class in NLP Amanda Cercas Curry, Zeerak Talat, Dirk Hovy. 2024. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) Association of Computational Linguistics.
[Conference Paper]
Subjective Isms? On the Danger of Conflating Hate and Offence in Abusive Language Detection. Amanda Cercas Curry, Gavin Abercrombie, Zeerak Talat. 2024. Proceedings of the 8th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Conference Paper]
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon. Fajri Koto, Tilman Beck, Zeerak Talat, Iryna Gurevych, Timothy Baldwin. 2024. The Proceedings of the 18th Conference of the European Chapter of the Association of Computational Linguistics. Association of Computational Linguistics.
[Conference Paper]
2023
Back to the Future: On Potential Histories in NLP Zeerak Talat, Anne Lauscher 2023. ArXiv
[Permaprint]
Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing. Lucie-Aimée Kaffee, Arnav Arora, Zeerak Talat, Isabelle Augenstein. 2023. Findings of the Association for Computational Linguistics: EMNLP 2023. Association of Computational Linguistics.
[Conference Paper]
Mirages. On Anthropomorphism in Dialogue Systems Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar, Verena Rieser, Zeerak Talat. 2023. The Proceedings of the 2023 Conference on Empircal Methods in Natural Language Processing. Association of Computational Linguistics.
[Conference Paper]
Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms Organizers of Queer in AI, Nathan Dennler, Anaelia Ovalle, Ashwin Singh, Luca Soldaini, Arjun Subramonian, Huy Tu, William Agnew, Avijit Ghosh, Kyra Yee, Irene Font Peradejordi, Zeerak Talat, Mayra Russo, Jess de Jesus de Pinho Pinhal. 2023. Proceedings of the Conference on Artificial Intelligence, Ethics, and Society
[Conference Paper]
It's Incomprehensible: On Machine Learning and Decoloniality. Abeba Birhane, Zeerak Talat. 2023. Handbook of Critical Studies of Artificial Intelligence. Edward Elgar Publishers.
[Book Chapter]
Proceedings of the 7th Workshop on Online Abuse and Harms (WOAH 2023). 2023. Proceedings of the 7th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Proceedings]
Futures for Research on Hate Speech in Online Social Media Platforms. Jaime Lee Kirtz, Zeerak Talat. 2023. Challenges and Perspectives of Hate Speech Analysis. Digital Communication Research.
[Book Chapter]
Federated Learning for Hate Speech Detection. Jay Gala, Jash Mehta, Deep Gandhi, Zeerak Talat. 2023. The Proceedings of the 17th Conference of the European Chapter of the Association of Computational Linguistics. Association of Computational Linguistics.
[Conference Paper]
[Best Paper Award] Queer In AI: A Case Study in Community-Led Participatory AI Organisers of Queer in AI 2023. FAccT '23: 2023 ACM Conference on Fairness, Accountability, and Transparency. Association of Computing Machinery.
[Conference Paper]
2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. Teven Le Scao, et al. 2022.
[Journal Paper]
A Federated Approach to Predicting Emojis in Hindi Tweets. Deep Gandhi, Jash Mehta, Nirali Parekh, Karan Waghela, Lynette D'Mello, Zeerak Talat. 2022. The Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association of Computational Linguistics.
[Conference Paper]
Directions for NLP Practices Applied to Online Hate Speech Detection. Paula Fortuna, Monica Dominguez, Leo Wanner, Zeerak Talat. 2022. The Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association of Computational Linguistics.
[Conference Paper]
Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models. Paul Röttger, Haitham Seelawi, Debora Nozza, Zeerak Talat, Bertie Vidgen. 2022. The Proceedings of the 6th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Paper]
Data Governance in the Age of Large-Scale Data-Driven Language Technology. Yacine Jernite, Huu Nguyen, Stella Biderman, Anna Rogers, Maraim Masoud, Valentin Danchev, Samson Tan, Alexandra Sasha Luccioni, Nishant Subramani, Isaac Johnson, Gérard Dupont, Jesse Dodge, Kyle Lo, Zeerak Talat, Dragomir Radev, Aaron Gokaslan, Somaieh Nikpoor, Peter Henderson, Rishi Bommasani and Margaret Mitchell. 2022. FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency. Association of Computing Machinery.
[Conference Paper]
On the Machine Learning of Ethical Judgments from Natural Language. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, Adina Williams. 2022. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association of Computational Linguistics.
[Conference Paper]
You Reap What You Sow: On the Challenges of Bias Evaluation Under Multilingual Settings. Zeerak Talat, Aurélie Névéol, Stella Biderman, Miruna~Clinciu, Manan Dey, Shayne Longpre, Alexandra Sasha Luccioni, Maraim Masoud, Margaret Mitchell, Dragomir Radev, Shanya Sharma, Arjun Subramonian, Jaesung Tae, Samson Tan, Deepak Tunuguntla, Oskar van der Wal. 2022. Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models. Association of Computational Linguistics.
[Workshop Paper]
Proceedings of the 6th Workshop on Online Abuse and Harms (WOAH 2022). 2022. Proceedings of the 6th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Proceedings]
2021
Disembodied Machine Learning: On the Illusion of Objectivity in NLP Zeerak Talat, Dilan Lulz, Joachim Bingel, Isabelle Augenstein 2021. ArXiv
[Permaprint]
A Survey of Race, Racism, and Anti-Racism in NLP. Anjalie Field, Su Lin Blodgett, Zeerak Talat, Yulia Tsvetkov. 2021. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association of Computational Linguistics.
[Conference Paper]
HateCheck: Functional Tests for Hate Speech Detection Models. Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Talat, Helen Margetts, Janet Pierrehumbert. 2021. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association of Computational Linguistics.
[Conference Paper]
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection. Bertie Vidgen, Tristan Thrush, Zeerak Talat, Douwe Kiela. 2021. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association of Computational Linguistics.
[Conference Paper]
Dynabench: Rethinking Benchmarking in NLP. Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Talat, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, Adina Williams. 2021. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association of Computational Linguistics.
[Conference Paper]
Findings of the WOAH 5 Shared Task on Fine Grained Hateful Memes Detection. Lambert Mathias, Shaoliang Nie, Aida Mostafazadeh Davani, Douwe Kiela, Vinodkumar Prabhakaran, Bertie Vidgen, Zeerak Talat. 2021. Proceedings of the 5th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Paper]
"Hold on honey, men at work": A semi-supervised approach to detecting sexism in sitcoms. Smriti Singh, Tanvi Anand, Arijit Ghosh Chowdhury, Zeerak Talat. 2021. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop. Association of Computational Linguistics.
[Workshop Paper]
Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021). Aida Mostafazadeh Davani, Douwe Kiela, Mathias Lambert, Bertie Vidgen, Vinodkumar Prabhakaran, Zeerak Talat. 2021. Proceedings of the 5th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Proceedings]
2020
Leaky academia: digital intimacy and open secrets in times of COVID-19. Nanna Thylstrup, Zeerak Talat, Daniela Agostinho. 2020. Identities - Global Studies in Culture and Power. Taylor and Francis.
[Workshop Paper]
Detecting East Asian Prejudice on Social Media. Bertie Vidgen, Austin Botelho, David Broniatowski, Ella Guest, Matthew Hall, Helen Margetts, Rebekah Tromble, Zeerak Talat, Scott Hale. 2020. Proceedings of the 4th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Paper]
Online Abuse and Human Rights: WOAH Satellite Session at RightsCon 2020. Vinodkumar Prabhakaran, Zeerak Talat, Seyi Akiwowo, Bertie Vidgen. 2020. Proceedings of the 4th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Paper]
Proceedings of the 4th Workshop on Online Abuse and Harms (WOAH 2020). Seyi Akiwowo, Bertie Vidgen, Vinodkumar Prabhakaran, Zeerak Talat. 2020. Proceedings of the 4th Workshop on Online Abuse and Harms. Association of Computational Linguistics.
[Workshop Proceedings]
2019
Proceedings of the 2019 Workshop on Widening NLP. Amittai Axelrod, Diyi Yang, Rossana Cunha, Samira Shaikh, Zeerak Talat. 2019. Proceedings of the 3th Workshop on Abusive Language Online. Association of Computational Linguistics.
[Workshop Proceedings]
Proceedings of the 3rd Workshop on Abusive Language Online. Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, Zeerak Talat. 2019. Proceedings of the 3th Workshop on Abusive Language Online. Association of Computational Linguistics.
[Workshop Proceedings]
2018
Proceedings of the 2nd Workshop on Abusive Language Online. *Darja Fišer, Ruihong Huang, Vinodkumar Prabhakaran, Rob Voigt, Zeerak Talat, Jacqueline Wernimont. 2018. Proceedings of the 2nd Workshop on Abusive Language Online. Association of Computational Linguistics.
[Workshop Proceedings]
Bridging the Gaps: Multi Task Learning for Domain Transfer of Hate Speech Detection. Zeerak Talat, James Thorne, Joachim Bingel. 2018. Online Harassment Springer.
[Book Chapter]
2017
Understanding Abuse: A Typology of Abusive Language Detection Subtasks. Zeerak Talat, Thomas Davidson, Dana Warmsley and Ingmar Weber. 2017. Proceedings of the First Workshop on Abusive Language Online. Association of Computational Linguistics.
[Workshop Paper]
Proceedings of the 1st Workshop on Abusive Language Online Zeerak Talat, Wendy Hui Kyong Chung, Dirk Hovy, and Joel Tetreault. 2017. Proceedings of the First Workshop on Abusive Language Online. Association of Computational Linguistics.
[Workshop Proceedings]
2016
Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter. Zeerak Talat. 2016. Proceedings of the First Workshop on NLP and Computational Social Science. Association of Computational Linguistics.
[Workshop Paper]
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. Zeerak Talat, Dirk Hovy. 2016. Proceedings of the NAACL Student Research Workshop. Association of Computational Linguistics.
[Workshop Paper]