No Culture Left Behind: ArtELingo-28, a benchmark of Wikiart with Captions in 28 Languages

Youssef Mohamed, Runjia Li, Ibrahim Said Ahmad, Kilichbek Haydarov, Philip Torr, Kenneth Ward Church, Mohamed Elhoseiny

EMNLP 24

---

• Abstract
• Paper
• Code and Dataset
• Citation

---

Abstract

Research in vision and language has made considerable progress thanks to benchmarks such as COCO. COCO captions focused on unambiguous facts in English; ArtEmis introduced subjective emotions and ArtELingo introduced some multilinguality (Chinese and Arabic). However we believe there should be more multilinguality. Hence, we present ArtELingo-28, a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations (140 annotations per image). Traditionally, vision research focused on unambiguous class labels, whereas ArtELingo-28 emphasizes diversity of opinions over languages and cultures. The challenge is to build machine learning systems that assign emotional captions to images. Baseline results will be presented for three novel conditions: Zero-Shot, Few-Shot and One-vs-All Zero-Shot. We find that cross-lingual transfer is more successful for culturally-related languages.

Paper

paper ArXiv 

Code and Dataset

Web Page:ArtELingo-28 

Citation

@inproceedings{mohamed-etal-2024-culture,
      title = "No Culture Left Behind: {A}rt{EL}ingo-28, a Benchmark of {W}iki{A}rt with Captions in 28 Languages",
      author = "Mohamed, Youssef  and
            Li, Runjia  and
            Ahmad, Ibrahim Said  and
            Haydarov, Kilichbek  and
            Torr, Philip  and
            Church, Kenneth  and
            Elhoseiny, Mohamed",
      editor = "Al-Onaizan, Yaser  and
            Bansal, Mohit  and
            Chen, Yun-Nung",
      booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
      month = nov,
      year = "2024",
      address = "Miami, Florida, USA",
      publisher = "Association for Computational Linguistics",
      url = "https://aclanthology.org/2024.emnlp-main.1165/",
      doi = "10.18653/v1/2024.emnlp-main.1165",
      pages = "20939--20962",}