Comparing Approaches to Dravidian Language Identification

Tommi Jauhiainen, Tharindu Ranasinghe, Marcos Zampieri

    Research output: Chapter in Book/Published conference outputConference publication

    Abstract

    This paper describes the submissions by team HWR to the Dravidian Language Identification (DLI) shared task organized at VarDial 2021 workshop. The DLI training set includes 16,674 YouTube comments written in Roman script containing code-mixed text with English and one of the three South Dravidian languages: Kannada, Malayalam, and Tamil. We submitted results generated using two models, a Naive Bayes classifier with adaptive language models, which has shown to obtain competitive performance in many language and dialect identification tasks, and a transformer-based model which is widely regarded as the state-of-the-art in a number of NLP tasks. Our first submission was sent in the closed submission track using only the training set provided by the shared task organisers, whereas the second submission is considered to be open as it used a pretrained model trained with external data. Our team attained shared second position in the shared task with the submission based on Naive Bayes. Our results reinforce the idea that deep learning methods are not as competitive in language identification related tasks as they are in many other text classification tasks.
    Original languageEnglish
    Title of host publicationProceedings of the Eighth Workshop on NLP for Similar Languages, Varieties and Dialects
    PublisherAssociation for Computational Linguistics (ACL)
    Pages120-127
    Number of pages8
    Publication statusPublished - Apr 2021

    Bibliographical note

    Copyright 2021 Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.

    Fingerprint

    Dive into the research topics of 'Comparing Approaches to Dravidian Language Identification'. Together they form a unique fingerprint.

    Cite this