Verified, then logged.
Sun 22 Nov 2026 · 00:20 KST / 15:20 UTC한국어 | ENSearch /

Breaking01 / AI

Medical AI shows gap between recall and evidence use

A Chinese-language benchmark reported lower scores for matching evidence and formulating recommendations than for identifying clinical information.

A clinician seen from behind reviews patient laboratory results alongside reference material and an unfinished AI recommendation at a hospital workstation, with all screen content unreadable. AI-generated image

Medical AI's knowledge of guidelines did not ensure patient-specific use of evidence, researchers reported in a paper published on 11 Nov 2026. Model rankings also changed across evaluation levels.

According to the paper, Med Evidence Bench (MEB), a clinical reasoning benchmark, contains 1,536 physician-written questions. It assesses guideline knowledge, standardised clinical scenarios and real-world laboratory scenarios. Each scoring criterion containing a medical claim is linked to a passage in a specified version of its source.

In real-world laboratory scenarios, the paper reported a mean score of 0.770 for identifying clinical information. The score for matching evidence was 0.165. The score for formulating recommendations was 0.160.

The researchers compared automated grading with physicians' majority judgements. They reported a macro F1 score, a classification measure used to assess agreement, of 0.930. Agreement was lower for judgements about applying evidence.

The paper's findings concern an evaluation based on Chinese-language materials. They show a gap between recalling guidelines and using evidence for individual patients. The evaluation does not establish effectiveness in clinical care.