BOISHOMMO: Benchmarking Class Imbalance in Bangla Multi-Label Hate Speech Detection

DOI

10.2991/978-94-6239-664-7_36How to use a DOI?

Keywords

Class Imbalance Analysis; Multi-label classification; Social media annotation; Low-resource NLP; Morphologically rich language; Inter-annotator agreement

Abstract

Class imbalance is an endemic and problematic issue in the Natural Language Processing (NLP) field, especially when Natural Language Processing is applied to low-resource languages (LRLs), in which annotated corpora often do not reflect the biased distributions of real-world hate speech. In an attempt to reduce this shortcoming, we introduce BOISHOMMO, a multi-labeled Bangali dataset specifically made available to enable the benchmarking of the agglutinative language-based class imbalance phenomena in a language with a population of over 250m; spoken by over 250 million people. The social-media comments contained in BOISHOMMO are annotated by native speakers on 2,499 comments, in ten categories that overlap, as follows: Race, Behaviour, Physical, Class, Religion, Disability, Ethnicity, Gender, Sexual Orientation, and Political. The final annotation process was done by majority voting and stabilized by both the Cohen and Fleiss Kappa statistics to guarantee that there was inter-rater consistency. In order to demonstrate the usefulness and complexity of the dataset, we performed the baseline classification experiments with the use of Logistic Regression and Linear Support Vector Classifiers (Linear SVC). The best performance achieved a Macro F1 -score of only 0.2101, which proved the existence of strong performance differences among majority categories (e.g., Behaviour) and minority categories (e.g., Disability). These numerical results support the idea that BOISHOMMO represents a challenging standard of testing the algorithms that are sensitive to class imbalance. It is expected that the publishing of this dataset will contribute to future studies in the fairness-conscious content moderation and multi-label classification of Indic languages.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Showrov Azam
AU  - Sifat Khan
AU  - Rashed Hossain
AU  - Nadim Mahmud
AU  - Md Abdullah Al Kafi
PY  - 2026
DA  - 2026/06/08
TI  - BOISHOMMO: Benchmarking Class Imbalance in Bangla Multi-Label Hate Speech Detection
BT  - Proceedings of the International Conference on Intelligent Data Analysis and Applications (IDAA 2025)
PB  - Atlantis Press
SP  - 516
EP  - 532
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6239-664-7_36
DO  - 10.2991/978-94-6239-664-7_36
ID  - Azam2026
ER  -