01 / AI · Analysis
CaDO automates answer repair in causal reasoning datasets
The researchers turned plain-language questions into queries they could verify and compared how often each method changed answers that were already correct.

Researchers reported in a paper that CaDO, a tool for repairing causal reasoning datasets, outperformed the strongest existing data-cleaning method in 16 of 18 experimental settings.
The researchers identified incorrect answers and an uneven mix of question types in datasets that ask about cause and effect. They also compared how often the repair methods changed answers that were already correct.
CaDO starts by turning plain-language questions into formal queries about cause and effect that can be checked, the paper explains.
Reduction in incorrect changes to correct CLadder labels versus an LLM-based judge
| CLadder | 92.2 |
|---|
Checks on individual questions and whole datasets
The paper assigns independent review agents to look for errors in individual questions.
At the dataset level, the researchers check which types of questions are covered and how the answer labels are distributed.
CaDO applies causal rules to repair questions with errors. Separate agents work together to generate new questions.
Preserving correct answers during repair
On the CLadder dataset, CaDO changed fewer already-correct answers than an evaluator based on a large language model, the researchers reported.
The paper's abstract does not disclose how many correct answers each method changed in error.