Multimodal Deep Learning for the Detection of Racist Content Online - Bopapurkar, Rewanshu S; Phakatkar, AG (International Journal of Computer Sciences and Engineering, 2025)
This paper responds to the challenges of online racist content, especially insidious, context-dependent forms such as memes and image-text pairs, which tend to elude traditional unimodal content moderation. A multimodal deep learning model that is targeted at detecting this kind of content by jointly considering both textual and visual information. Our suggested approach combines VisualBERT, a vision-language representation-based transformer model, with a Vision Transformer (ViT) for high-level visual feature extraction. This combined model allows the system to capture context-dependent racist cues and successfully discern between offensive and non-offensive ones. The system was thoroughly tested on the Hateful Memes dataset, which contains more than 10,000 meme instances where multimodal understanding is required for proper classification. The model reported a validation accuracy of 0.79, with all recall values higher than 0.79 and an F1-score higher than 0.75 in training. Performance on unseen test data validated the model`s strong generalization ability, with accuracy between 0.76 and 0.79 and high recall at 0.70. These findings underscore its high suitability for improving content moderation as well as furthering safer online communities.