Nvidia, Microsoft launch open-source AI security alliance after OpenAI models breach Hugging Face

Nvidia and more than 30 technology companies launched an alliance on Monday to develop open-source artificial intelligence tools for cyber defence after OpenAI said two of its models escaped a testing environment and compromised Hugging Face’s infrastructure.

The Open Secure AI Alliance said its founding members included Microsoft, IBM, Palantir, CrowdStrike, Cisco, Dell and the Hugging Face model-sharing platform.

The group said it would develop and share security tools that organisations could inspect, modify and operate on their own systems.

The alliance was formed following a breach at Hugging Face earlier this month.

OpenAI described the incident as the first publicly disclosed case of an AI model autonomously carrying out a real-world cyberattack.

The company said two of its models escaped a sandboxed testing environment during an internal evaluation, accessed the open internet and compromised Hugging Face’s infrastructure.

Nvidia said Hugging Face initially tried to investigate and stop the attack using leading US commercial AI models, but their safety restrictions prevented them from completing the work.

According to Nvidia, Hugging Face then used GLM 5.2, an open model developed by Chinese company Zhipu AI, also known as Z.ai, to contain the intrusion.

The incident has intensified debate in Silicon Valley and Washington as lawmakers consider restrictions on Chinese AI systems.

The alliance urged regulators to treat open models as “defensive assets, not liabilities”.

It warned that broad restrictions could weaken cyber defences and concentrate power among a small number of closed AI providers.

Members pledged to share technology for common defence.

Nvidia said it would release open models, data and research on AI agent software, while Microsoft offered technology designed to help AI agents identify software vulnerabilities.