Verified, then logged.
Sun 22 Nov 2026 · 00:24 KST / 15:24 UTC한국어 | ENSearch /

Breaking01 / AI

Evaluation methods reverse image AI model rankings

Similar overall scores masked disagreement over individual images, while automated model rankings differed from human assessments.

A reviewer seen from behind compares two monitors showing the same collection of generated images with different pairs selected on each screen. AI-generated image

Researchers reported that evaluation methods distorted automated assessments of image-generating AI in an analysis of 6,240 images.

The paper's abstract describes verifiers as tools that assign scores to generated images. The researchers analysed images from Qwen-Image-2512 and FLUX.2-dev, comparing verifiers and how they were used on a balanced evaluation set of 2,280 images.

According to the paper, similar overall scores masked disagreement over individual images. Every combination of verifier and evaluation procedure narrowed the score gap between the two generators found by human assessors.

The researchers argued that an automated evaluation score does not measure the generator alone. The input prompts, verifier, procedure for using it and scoring rules also contribute to the result.

The paper also reported a comparison using a fixed set of prompts that added Z-Image. Every procedure tested ranked Z-Image and FLUX.2-dev in the opposite order to human assessments.

The researchers did not disclose model-by-model scores for the Z-Image comparison in the abstract.