Verified, then logged.
Sun 22 Nov 2026 · 00:19 KST / 15:19 UTC한국어 | ENSearch /

Breaking01 / AI

Language gaps persist in document understanding benchmark

Researchers reported that larger models did not narrow performance gaps between languages in the MDU benchmark.

From behind and below the waist, a person with reflective markers on their knees and ankles walks along a force-plate walkway in a clinical gait lab lined with tripod-mounted motion-capture cameras. AI-generated image

Matteo Gabburo and colleagues introduced MDU, a document understanding benchmark covering 13 tasks, to measure AI performance across languages.

According to the paper's abstract, MDU contains 23,146 evaluation examples in English, French, German, Italian and Spanish. Tasks include text extraction, summarisation and identifying document layouts. The arXiv submission date was 10 Nov 2026.

The researchers evaluated vision-language models, which process images and text. They tested 45 open-weight models, whose trained model weights are publicly available, alongside a commercial system using the same procedure.

The highest average score was 69.0, a researcher-published benchmark figure in the abstract. The researchers reported that scores across tasks were largely uncorrelated after accounting for overall model performance, meaning strength in one task did not consistently accompany strength in another.

In their comparison of model sizes, the researchers reported that the largest models did not achieve the best results. The performance gap between each model's strongest and weakest language also did not narrow as models grew larger.

The researchers did not disclose model names or individual scores in the abstract.