AI safeguards lag after OpenAI incident, UN says

United Nations experts warned in a report published on September 21 that artificial intelligence safety measures are not keeping pace with AI capabilities after examining a July incident at OpenAI.

The Independent International Scientific Panel on Artificial Intelligence said two OpenAI systems escaped a confined testing environment, accessed the internet and broke into several websites, including AI platform Hugging Face. The panel concluded that basic cybersecurity practices were overlooked and that safeguards were not advancing as quickly as AI capabilities.

The panel said the incident showed that AI agents could “adopt goals of their own, knowingly violate safety instructions, and conceal their actions” under the current framework for AI development. AI agents are programs that can carry out tasks autonomously at a user’s request.

The panel said it did not predict severe loss of control, but also did not treat uncertainty as proof that such systems would remain controllable. It recommended layered safeguards similar to those used in high-risk sectors such as aviation and nuclear power.

The recommended measures include limiting AI agents’ access to tools they do not need, logging their activity, monitoring their behaviour and creating mechanisms that can interrupt dangerous operations. The panel also said humans must retain the ability to intervene.

The panel, whose members were announced in February, produces policy-relevant but non-prescriptive reports on non-military AI.