Thought Leadership
The open-source platform Hugging Face has fallen victim to an attack carried out by an autonomous AI agent. Internal datasets and credentials were among the information affected.
The AI platform Hugging Face has disclosed a security incident in its production infrastructure that was carried out by an autonomous AI agent system. The company detected the unauthorized access last week.
“We detected unauthorized access to a limited selection of internal datasets and several credentials used by our services.”
Hugging Face
Based on the investigation so far, there is no evidence that publicly accessible models, datasets, Spaces, or the software supply chain were tampered with. The attack began in the data processing pipeline. A malicious dataset exploited two code execution paths: a remote code dataset loader and a template injection in a dataset configuration.
This allowed code to run on a processing system. From there, the attacker gained node-level access, harvested cloud and cluster credentials, and moved laterally across multiple internal clusters over the weekend. The attack was carried out by an autonomous agent framework that performed thousands of individual actions in ephemeral sandbox environments. Its command-and-control infrastructure migrated autonomously across public services.
Mitigation and Remediation Measures
The exploited code execution paths have been shut down by the operator. The following remediation measures have been implemented:
- Removed access within the affected clusters and rebuilt the compromised nodes
- Revoked and rotated affected credentials and tokens, along with a precautionary broader rotation of system secrets
- Introduced additional security controls and stricter admission policies across the clusters
- Improved detection and alerting to enable around-the-clock notification within minutes
Hugging Face is urging customers to rotate existing access tokens as a precaution and review their recent account activity.
AI Security Filters Complicate Forensic Analysis
During its investigation, the company encountered unexpected obstacles caused by security mechanisms in Western AI models. These systems refused to process requests containing real attack commands, exploit payloads, or artifacts from the attackers’ infrastructure. The commercial models’ safety filters were unable to distinguish between an actual attack and legitimate incident response activities. Hugging Face therefore used Z.ai’s Chinese open-weight model GLM 5.2 to conduct its forensic analysis.
“This experience points to a gap that organizations should plan for,” the company said. “We do not know what model powered the attacker’s agents, whether it was a jailbroken hosted model or an unrestricted open-weight model. Either way, the attacker was not bound by any usage policy, while our own forensic work was blocked by the safety filters of the hosted models we initially tried.”
The incident has also prompted Hugging Face to draw technical lessons for IT departments:
“The practical lesson for defenders: Have a capable model available before an incident that you can run and verify on your own infrastructure, both to avoid being blocked by safety filters and to prevent the attacker’s data and credentials from leaving your environment.”
(ll)
