01 / AI · Claim vs. Check · Analysis
KLEAR holds knowledge fixed to test Korean LLMs' application
The design aims to separate the causes of wrong answers that accuracy scores conceal, though the abstract names no evaluated models and gives no scores.

KLEAR, a framework for evaluating Korean large language models (LLMs) posted on arXiv on 9 Nov 2026, holds the required knowledge fixed and raises only the difficulty of applying it. According to the paper's abstract, the design aims to separate wrong answers caused by missing knowledge from those caused by failed application.
The researchers argued that an accuracy score alone cannot show whether a model lacked the knowledge or failed to use what it knew. KLEAR moves that distinction into the design stage of the test questions.
The researchers first examined what existing Korean tests actually measured.
Existing tests that factual recall can solve
The researchers wrote in the abstract that most questions in existing Korean benchmarks can be answered through "factual recall alone". A model can reach the answer simply by remembering a fact.
Questions that demanded more application also differed in the knowledge they required, according to the researchers' analysis. That made it difficult to isolate application ability from the difference in scores between two questions.
Harder questions built on the same knowledge
According to the paper, KLEAR sets out the knowledge each original question requires as an explicit anchor bundle. It uses that bundle to generate questions requiring more complex application. The result is a contrast set with the same knowledge and different application difficulty.
The researchers said they put the generated candidate questions through automated and human validation and revised them repeatedly using evaluation feedback. In experiments, the finished sets largely met the intended constraints. The abstract does not give the share of questions that fell outside those constraints.
Application failures that persist with knowledge provided
The researchers said they evaluated a range of LLMs on the knowledge-matched sets and that raising application difficulty exposed model failures. Those failures persisted "even when the required knowledge is provided", the abstract states.
The abstract does not name the models evaluated or give scores for each model. Which models are weak at application depends on those figures. The question KLEAR asks is why a model got an answer wrong, not how many points it scored.