Verified, then logged.
Sun 22 Nov 2026 · 00:22 KST / 15:22 UTC한국어 | ENSearch /

01 / AI · Analysis

Multi-word training cuts English tokens; model tests remain

Researchers reported compression gains but did not evaluate models trained with the new tokenizers.

Seen from behind in a Phnom Penh office, a person compares several blurred columns of written answers on a laptop, with a Cambodian temple roof visible through the window. AI-generated image

Researchers reported that Havon v4 used 6.1% fewer English tokens than GPT-4o's tokenizer on FLORES-200. The comparison matched vocabulary sizes and measured compression, excluding evaluation of models trained with the tokenizer.

The paper highlighted a gap in token counts between Indian languages and English. The researchers explained that processing the same content in Indian languages required more tokens, making it more expensive and slower to serve.

The researchers added a multi-word training stage to Havon v4, allowing it to group several words together. They compared the compression gains across languages, leaving model evaluation as a separate task.

Model evaluation remains beyond compression

The researchers limited their findings to compression experiments. The paper reports no evaluation of a model trained with Havon v4 or v5.

The paper compared tokenizers with the same vocabulary size as GPT-4o's tokenizer. At that size, the researchers reported that Havon v3 used more English tokens, while Havon v4 reduced the count.

Savings vary across writing systems

The paper reported that multi-word training produced its largest gains in English, Hindi and the Romance languages. The compression benefit therefore varied across languages.

The researchers also identified limits in writing systems without spaces between words. They reported smaller improvements for Khmer, Burmese and Tibetan.

For Havon v5, the researchers changed the code training mix and reported lower code token counts, although these remained above GPT-4o's. Those results extended the compression comparison; evaluation of models trained with the new tokenizers remained outstanding in the paper.