Verified, then logged.
Sun 22 Nov 2026 · 00:21 KST / 15:21 UTC한국어 | ENSearch /

Breaking01 / AI

CaseScope finds instability in AI pathology diagnoses

Open models were more affected by specimen-site information than by additional images on teaching cases, the researchers reported.

A long-exposure photo in a dark physics lab shows three small lamps of unequal size tracing tangled orbital light trails above a black table, with one trail breaking away toward the edge of the frame. AI-generated image

CaseScope researchers reported unstable pathology diagnoses in their 13 Nov preprint, CaseScope: A Multi-Field Benchmark for Case-Level Pathology Diagnosis with Multimodal LLMs: the best open model remained correct across every answer-option rotation on 17.9% of cases (n=436, teaching cases, all option rotations). The study, which evaluates multiple views of each lesion, is not peer-reviewed and reports no independent replication results.

The researchers grouped images by lesion from a residency-training image library and published pathology atlases. Teaching cases were aligned to a common classification system, with a pathologist reviewing cases flagged by an automated preliminary check. Atlas cases were accepted only when language models reading the captions agreed on their classification.

The best proprietary model remained correct across every answer-option rotation on 61.7% of cases (n=436, teaching cases, all option rotations), the researchers reported. This measure requires a correct answer regardless of how the options are arranged.

On teaching cases, the researchers reported that open models made little use of additional images. Specimen-site information had a greater effect under these conditions, and wrong answers tended towards diseases with a characteristic resemblance to the correct diagnosis.

The authors described earlier evaluations as focusing mainly on single-image questions or systems that process entire tissue slides. CaseScope instead asks models to combine several views of the same lesion.

The researchers reported that evaluation code is available, but the abstract provides no independent replication results. It also does not name the individual models evaluated or specify their execution settings.