Breaking01 / AI
DramaGrow reports accuracy gap between AI judging methods

The authors of DramaGrow, a paper submitted on 14 Oct 2026 and not yet peer-reviewed, report that direct pairwise comparison achieves 85.1% accuracy (n=114, fixed preference pairs) in tests comparing AI judging methods on Chinese serial dramas; its abstract reports no independent replication.
The authors distinguish verifiable violations of requirements from human preferences and disagreements among human evaluators. They apply separate evaluation criteria to requirement violations and preference judgements.
Repeated scoring yields mean accuracy of 53.8–55.8% (n=114, fixed preference pairs), according to the paper. DirectPair-GLM, which compares dramas directly in pairs, achieves the higher accuracy reported above on the same preference pairs.
The authors argue that the range of scores treated as a tie, variation between model calls and the margin between human votes affect performance. The abstract reports no evaluation results for serial dramas in other languages.
According to the authors, most existing evaluations compress different kinds of quality judgement into a score from a single model call. DramaGrow instead applies separate criteria to requirement violations, preferences and disagreements among evaluators.
The authors plan to release the evaluation protocol, anonymised human evaluations, model evaluation records and analysis code. The abstract gives no release date.