OffensEval 2023: Offensive language identification in the age of Large Language Models

Marcos Zampieri, Sara Rosenthal, Preslav Nakov, Alphaeus Dmonte, Tharindu Ranasinghe

Research output: Contribution to journalReview articlepeer-review


The OffensEval shared tasks organized as part of SemEval-2019–2020 were very popular, attracting over 1300 participating teams. The two editions of the shared task helped advance the state of the art in offensive language identification by providing the community with benchmark datasets in Arabic, Danish, English, Greek, and Turkish. The datasets were annotated using the OLID hierarchical taxonomy, which since then has become the de facto standard in general offensive language identification research and was widely used beyond OffensEval. We present a survey of OffensEval and related competitions, and we discuss the main lessons learned. We further evaluate the performance of Large Language Models (LLMs), which have recently revolutionalized the field of Natural Language Processing. We use zero-shot prompting with six popular LLMs and zero-shot learning with two task-specific fine-tuned BERT models, and we compare the results against those of the top-performing teams at the OffensEval competitions. Our results show that while some LMMs such as Flan-T5 achieve competitive performance, in general LLMs lag behind the best OffensEval systems.
Original languageEnglish
Pages (from-to)1416 - 1435
Number of pages20
JournalNatural Language Engineering
Issue number6
Publication statusPublished - 6 Dec 2023

Bibliographical note

© The Author(s), 2023. This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited


  • Machine learning
  • Text classification


Dive into the research topics of 'OffensEval 2023: Offensive language identification in the age of Large Language Models'. Together they form a unique fingerprint.

Cite this