Verified, then logged.
Sun 22 Nov 2026 · 00:33 KST / 15:33 UTC한국어 | ENSearch /

Breaking01 / AI

Structured records flag the same missing results as answers

The author drew no conclusion about the effect of adding a record after the evaluator fell short of a pre-set human validation standard on the key category.

A lawyer, seen from behind in a law office at dusk, reviews blurred contract redlines on a laptop at a long conference table. AI-generated image

The author reported in a preprint posted on 19 Nov 2026 that model answers flagged every missing lookup result flagged by their accompanying structured records in a comparison of synthetic logs (n=30, records flagging a missing result), and the paper reports no independent replication of that finding.

The paper, which has yet to be peer reviewed, describes how the author used Codex CLI to have gpt-5.6-sol write an answer and a structured record in the same response. Each log with a missing follow-up result was paired with a log in which the result came back (n=100, synthetic log pairs in the preregistered experiment).

The author reported that records flagged the missing result in 29% of cases (n=100, logs with no returned result, unresolved-items format) and 5% of cases (n=100, logs with no returned result, findings-only format).

In the author's exploratory analysis, all 30 records that flagged the missing result accompanied answers that flagged the same gap (n=30, records flagging the missing result). There were zero records that flagged the gap when the answer omitted it (n=70, answers omitting the missing result).

According to the author, earlier work tested an instruction asking the model to answer honestly. The new paper also tested asking the model to list unresolved items in a record alongside its answer.

The author reported no statistically significant difference in the answer evaluation against the instruction to answer honestly. The evaluator fell short of the preregistered human validation standard on the key category, so the author drew no conclusion about the effect of adding the record.