01 / AI · Analysis
Scientific code AI routes repairs by failure type
Targeted repairs and separate verification put token use and translation correctness under the same evaluation.

Researchers reported that a scientific code translation system used up to 52.4% fewer tokens than a comparison method in an evaluation using open-source models. The reduction is a researcher-published benchmark result.
The paper reported that separate verification exposed incorrect translations that passed compilation and execution checks. Its design distinguishes code that runs from code that has been translated correctly, linking the token savings to an additional assessment of correctness.
The system assigns translation, compilation diagnosis and execution validation to independent AI agents. A coordinating component uses failure verdicts to select the relevant repair function, while separate verification checks the resulting translations.
Repairs matched to failure types
The paper described existing language-model approaches as fixed sequences of tasks. It identified their use of the same repair strategy across different failures as a limitation.
The researchers designed a coordinator that selects a targeted repair function based on the verdict. An automated output check guides the cycle of corrections and retesting, making the failure assessment the basis for the next repair.
Verification after execution checks
The evaluation covered 30 computational tasks from HeCBench. The researchers translated them in both directions between CUDA, OpenMP and SYCL, programming frameworks for parallel computing.
The researchers reported higher verified correctness than the comparison method across all 18 combinations of model and translation direction. These are researcher-published benchmark results, and token use was also assessed against that method. The abstract does not give correctness rates for individual combinations.
According to the paper, separate language-model verification exposed incorrect results that compilation and execution checks had accepted. The system’s reported correctness therefore includes an additional judgement after execution, making that verification step part of how its results should be read alongside token use.