Verified, then logged.
Sun 22 Nov 2026 · 00:23 KST / 15:23 UTC한국어 | ENSearch /

01 / AI · Analysis

Anthropic proposes safety tests for whole conversations

Anthropic argues that AI safety evaluations should assess both compliance with harmful requests and excessive refusal of harmless ones.

Image from Anthropic
Image: Anthropic

AI company Anthropic proposes whole-conversation safety evaluations in Building strong safety evaluations, describing tests that use a separate Claude model to simulate users and reporting no independent replication results.

Anthropic says safety evaluations inform decisions about whether to deploy a model and which safeguards accompany it. Comparing scores across models, it argues, cannot by itself establish which intervention changed their safety.

The company recommends assessing whether responses remain appropriate throughout a conversation, including both compliance with harmful requests and excessive refusal of harmless ones. That shifts the focus from whether a model declines to answer to whether its response fits the circumstances.

What refusal rates miss

Anthropic's article distinguishes the appropriate response rate from the refusal rate. Its multi-turn evaluations assess whether a model responds appropriately throughout a conversation against criteria specific to each risk area. An inappropriate response at any point causes the whole conversation to fail.

For a user in crisis, Anthropic argues, refusing to engage is insufficient. Its evaluation criteria include whether the model responds with empathy and directs the user towards human support at an appropriate moment.

Simulated conversations from a shared starting point

Anthropic says domain experts define simulated users' objectives, tactics and conversational styles. Each scenario starts with a fixed prompt, after which a separate Claude model generates follow-up messages in response to the model being evaluated.

The company runs each scenario repeatedly with different combinations of conditions, then uses a grading model to assess the transcripts. It recommends varying factors such as the severity of a situation and the user's tone independently, so their effects can be examined separately.

The gap between simulated and real users

Anthropic acknowledges that simulated users are simpler than real people and that user behaviour changes over time. It says it continually adds and revises evaluation cases using behaviour observed in actual use.

The article reports no independent replication results for the proposed evaluation approach.

Compared with single-question evaluations, Anthropic argues, whole-conversation tests can account for how earlier remarks change the risk attached to a later request. Its recommendation is to evaluate that developing context as well as individual responses, while continuing to revise scenarios as real-world behaviour changes.