Claude's cyber tests reached real systems. What failed at the boundary?

Anthropic reported three incidents in which Claude models gained unauthorized access to real organizations during cyber evaluations. The models were assigned fictional capture-the-flag exercises and told they had no internet access. In fact, a third-party evaluation environment could reach the live internet. The first control to check in any agent test is the boundary the agent can actually cross, not the boundary described in its prompt.

In its July 30 disclosure, Anthropic said it reviewed 141,006 evaluation runs where internet access might have been possible. It found three incidents across six runs, affecting three organizations. The tests involved three different models: Claude Opus 4.7, Mythos 5, and an internal research model. They ran on dedicated evaluation infrastructure without the cyber classifiers and monitoring normally used for generally available models. Anthropic said that infrastructure had no access to its sensitive internal systems or customer data.

How did test work become real access?

The prompts framed each task as a simulation and did not define the systems that were in scope. In one case, the fictional target's name resembled a real company's domain. The model found the live site, accessed credentials and production data, and continued after signs that the system was real. In another, a model published a malicious Python package to the public PyPI registry while trying to solve the exercise. It ran on 15 real systems; one scanner leaked credentials that the model then used to reach further infrastructure. A third model found and compromised a public-facing application, then stopped when it concluded the target was unrelated to the exercise.

Anthropic said the models did not deliberately attempt to escape the test environment or copy themselves elsewhere. This does not soften the harm: real systems were accessed and third-party credentials were taken. It distinguishes an unintended open network path from a deliberate technical escape through an isolation flaw. In its July 30 report, Anthropic said its investigation and remediation work were continuing.

What should an evaluation owner do?

1. Step 01Verify network paths before the first run. Test direct access, tool-mediated access, package registries, and other routes to the public internet.
2. Step 02Define scope in operational terms. Name permitted targets and actions, and state what the agent must do when a target is unreachable or a real system appears.
3. Step 03Watch the run. Keep tool and network activity visible, with an operator who can pause it when the agent reaches an unexpected system.
4. Step 04Review the trace afterward. A completed test score does not reveal whether the model crossed a boundary along the way.

These controls apply to any powerful agent given tools and an open-ended goal. The test environment itself is part of the security system. Treat the configured permissions, reachable network, and recorded actions as evidence that must agree with the written task.