Breaking01 / AI
Language gaps persist in document understanding benchmark
Researchers reported that larger models did not narrow performance gaps between languages in the MDU benchmark.

Matteo Gabburo and colleagues introduced MDU, a document understanding benchmark covering 13 tasks, to measure AI performance across languages.
According to the paper's abstract, MDU contains 23,146 evaluation examples in English, French, German, Italian and Spanish. Tasks include text extraction, summarisation and identifying document layouts. The arXiv submission date was 10 Nov 2026.
The researchers evaluated vision-language models, which process images and text. They tested 45 open-weight models, whose trained model weights are publicly available, alongside a commercial system using the same procedure.
The highest average score was 69.0, a researcher-published benchmark figure in the abstract. The researchers reported that scores across tasks were largely uncorrelated after accounting for overall model performance, meaning strength in one task did not consistently accompany strength in another.
In their comparison of model sizes, the researchers reported that the largest models did not achieve the best results. The performance gap between each model's strongest and weakest language also did not narrow as models grew larger.
The researchers did not disclose model names or individual scores in the abstract.