Breaking01 / AI
PolyRoute sets benchmark for routing queries in 12 languages
All test messages and answers are AI-generated, while a statistical classifier scores 91.9% in the researchers’ published results.

Researchers introduced PolyRoute, a benchmark for models that route support queries to the appropriate department across 12 languages, in an abstract published on 11 Nov.
According to the abstract, the test set contains 322 native-language messages and 156 messages that mix languages or use Roman letters. All messages and their correct answers were generated by large language models.
The researchers’ published benchmark results put the released multilingual model at 77.3%. The best model adjusted through additional training scores 96.6%.
For comparison, the researchers’ published results give the English-based model 61.2%. A statistical classifier that uses word importance scores 91.9%.
The researchers said the test set is large enough to distinguish only score differences greater than about 4 percentage points. This limits its ability to separate models with smaller score differences.
The researchers did not disclose performance measured on real customer queries in the abstract.