02 / IT · Analysis
AI test answers can vary with the same code and prompt
A single AI test response gives quality teams little basis for judging consistency across repeated runs.

An article on DEV Community reported that AI testing agents can give different answers to the same request, even when the code is unchanged. It presented this as a problem for quality engineering teams that use agents to check their apps.
The article described teams requesting tests in everyday language. Receiving an answer once does not guarantee the same answer on a later run. That distinction matters when teams judge whether a test is repeatable.
The article's title framed agent workspaces as a way to make testing repeatable. This shifts the focus from an agent's ability to answer a request to its ability to repeat a test consistently.
Different answers from unchanged code
The article used a checkout test as its example. A team connects an agent to its app, then asks it to check the checkout flow and report problems.
According to the article, sending the same request the next day can produce a different answer. The example explicitly keeps the code unchanged, showing that an unchanged app and request do not guarantee a consistent response.
Workspaces and repeatable testing
The DEV Community article's title positioned agent workspaces as a means of making tests repeatable. That framing addresses the variation in answers described in its checkout example.
The problem it described suggests that getting an answer is only part of assessing an AI test. For teams delegating app checks to agents, the next consideration is whether repeated runs produce consistent answers. Request completion and test repeatability remain separate judgements.