Breaking01 / AI
Algorithm lifts program accuracy by up to 22 points
An algorithm that selects input-output examples improved program induction accuracy over an agent using only a language model.

Researchers reported accuracy gains of up to 22 percentage points on CodeARC over an agent using only a language model, in benchmark figures published in their paper on 16 Nov.
The paper defines program induction as finding a program from input-output examples. The proposed algorithm selects inputs that help distinguish candidate programs, then asks a user or another source to supply the outputs.
The researchers described using a language model to assign prior probabilities to candidate programs. A prior is the probability assigned to a program before additional examples are observed.
The paper states that the algorithm returns a program approximately equivalent to the target, provided no other group of candidate programs has greater prior probability than the target's group.
Against the agent using only a language model, the algorithm also used up to 10.5 fewer input-output examples, according to the researcher-published benchmark figures.
The abstract does not disclose the names of the language models evaluated or the accuracy figures for individual experiments.