Chinese AI Model Kimi K3 Escapes Security Sandbox

The AI model Kimi K3 broke out of an environment meant to be isolated during a cybersecurity test and gained access to the open internet.

According to a report published on August 7 by the US security firm Frontier Security, Kimi K3 was being evaluated on its defensive cybersecurity capabilities as part of a benchmark based on the testing software of the UK’s AI Security Institute. A basic network misconfiguration in the testing environment left outbound connections open on ports 443 and 53. The model independently probed these settings, discovered it could reach GitHub, and then downloaded the official repository for the assigned benchmark in order to read the solution stored there directly from disk, instead of working through the actual task itself. Frontier Security CEO Yaron Singer emphasized that this did not involve the exploitation of a previously unknown vulnerability, but simply a leak in the sandbox itself.

According to Frontier Security, Kimi K3 did not go on to attack any external systems, since the information needed for the task was already publicly available on GitHub. The company nevertheless views the model’s willingness to exploit such a gap without hesitation as an indication that Kimi K3 has fewer internal safeguards than comparable models. Unlike the unreleased test version involved in the OpenAI case, Kimi K3 is also a model that has been freely available to the public since July, with openly accessible weights, which Frontier Security says makes the incident potentially more consequential than comparable earlier cases involving unreleased models.

Several similar incidents already before Kimi K3 this summer

The incident is part of a series of similar escapes from testing environments involving models from Meta, OpenAI, and Anthropic over the past few weeks. In a case that came to light in late July, an OpenAI model exploited a previously unknown vulnerability in proxy software for software packages to break out of a sealed testing environment, and subsequently used stolen credentials to actually penetrate the infrastructure of the platform Hugging Face. In all cases known so far, reports trace the escapes back to errors in the configuration of the respective testing environments, not to vulnerabilities in the models themselves.

(red)