01 / AI · Analysis
Peer signals reduce LLM public accuracy by up to 47.8 points
The researchers wrote that multi-agent evaluations must measure whether agreement reflects private judgments, not only whether models reach it.

Ambiguous peer signals cut the public accuracy of nine large language models (LLMs) by as much as 47.8 percentage points, according to figures the researchers published in a multi-agent study. Public accuracy is the accuracy of the decision a model makes public after seeing other agents' signals. The paper, Phantom Consensus, was posted to arXiv on 9 Nov 2026.
Systems that pool the judgments of several model instances are increasing, the researchers wrote. They raised the concern that reasoning about other agents can itself cause failure. The study examines apparent consensus that the group's recorded private judgments do not support.
The researchers built a separate tool to measure that apparent consensus.
A benchmark for apparent consensus
The paper calls this failure phantom consensus: a model changes its public decision after seeing an ambiguous social signal. In these cases, the models inaccurately estimated what other agents had privately judged.
The abstract sets out the design of the benchmark, PHANTOM-Bench, step by step. It separates private judgments made in isolation, public decisions made after a signal, and estimates of the group's judgment. It also includes control conditions for neutral wording and explicit vote-following.
When correct private judgments became wrong conclusions
The nine models tested were a mix of closed and open-weight systems, the abstract says, with open-weight meaning that a model's trained parameters are publicly released. Ambiguous signals consistently degraded decision quality. Some groups that were correct in their private judgments reached wrong conclusions at the public stage.
In the most susceptible setting, collective rationality failures reached 47.3%, according to figures the researchers published. They also observed substantial errors in estimates of the group's private judgments alongside the public-stage failures. The abstract does not give before-and-after accuracy values or per-model figures.
Repairing failures without revealing the answer
In the control conditions, the paper said, susceptibility to ambiguous and explicit peer signals differed by model. Revealing the agents' recorded private decisions repaired many failures without disclosing the correct-answer labels. Recovery also varied by model, and the abstract gives no per-model recovery figures.
The abstract closes by saying that final agreement is not the only thing to measure. Evaluators must also check whether that agreement reflects "independently measured private judgments".