Verified, then logged.
Sun 22 Nov 2026 · 00:21 KST / 15:21 UTC한국어 | ENSearch /

Breaking01 / AI

Apple paper tests LLMs against humans on translation scoring

The paper argues that LLMs need thorough evaluation across language pairs and domains before they replace human evaluation of translations.

Seen past the blurred figure of a translation reviewer, a monitor displays two side-by-side columns of illegible text in different scripts with error spans highlighted in amber and red, while bilingual dictionaries lie open on the desk. AI-generated image

Apple's machine learning research site published a paper on 12 Oct 2026 that compared large language models (LLMs) with humans on two schemes for evaluating machine translation.

The schemes are Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA), according to the paper's abstract. Model performance was measured by agreement with humans.

The abstract says LLMs are considered a more efficient and cheaper alternative to human judgment in translation evaluation. That evaluation varies by language pair, domain and level of annotation detail. The authors write that LLMs must be thoroughly evaluated across those dimensions before they can reliably replace human evaluation.

The paper's title states that humans and models are both challenged by translation quality annotation.

The opening of the abstract does not give agreement figures or name the models tested.